Atlas

Compare Models

DeepSeek V4.1 Flash 1017 vs GPT-6 Sol

GPT-6 Sol leads by 2.8 capability points (≈95% interval +0.4 to +5.3): a clear lead over DeepSeek V4.1 Flash 1017 · P(higher) ≈ 99%

Wins 81 of 107 shared benchmarks (26 lost)

Capability vs. relative cost
DeepSeek V4.1 Flash 1017GPT-6 SolClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

DeepSeek V4.1 Flash 1017vsGPT-6 Solcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins26 – 81 · 107 shared

Comparing matching setups only (31 benchmarks excluded) · suites counted once
← DeepSeek V4.1 Flash 1017GPT-6 Sol →

DeepSeek V4.1 Flash 1017vsGPT-6 Solwins by categoryData Nov 15, 2026Data: Artificial Analysis, SecureBio, Vals AI and 27 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6 Sol minus DeepSeek V4.1 Flash 1017.

AA-PersonalityBench Organization indexChat & Writing88.612.4−76.9
CyberGym-E2E-AA (Artificial Analysis)Cybersecurity76.30.0−76.3
AA-PersonalityBench Informativeness indexChat & Writing71.410.5−61.5
HardGeoBench Avg DistanceMultimodal32011497+52.6
LisanBench Difficulty-WeightedGames77333118−49.4
ArXivMath 08/2026 (Native Harnesses)Math43.993.0+49.1
AA-PersonalityBench Transparency indexChat & Writing86.739.1−48.1
Cresta-BenchAgents32.677.9+45.3
AA-PersonalityBench Conviction indexChat & Writing94.350.5−44.2
FrontierMath Tier 4 (v2) - PrivateMath46.390.0+43.7
LisanBench Path LengthGames211078222−40.0
AA-PersonalityBench Slop indexChat & Writing39.11.0−38.5
BrokenArXiv 08/2026 (Native Harnesses)Math46.778.6+31.8
BuseyBench SVGMultimodal4.37.5+31.8
HardGeoBench Country AccuracyMultimodal15.046.3+31.3
CritPt v2 (agentic, avg@5)Science50.680.9+30.3
HardGeoBench <=100miMultimodal7.537.5+30.0
AA-PersonalityBench Warmth indexChat & Writing27.754.3+26.9
Humanity's Last Exam DiamondGeneral QA20.845.2+24.4
HardGeoBench <=10miMultimodal3.826.3+22.5
ALE-BenchCode20122637+22.1
Tax Agent BenchFinance & Accounting74.953.0−21.9
DeepsecBench-AA (Artificial Analysis)Cybersecurity24.946.3+21.4
AA-PersonalityBench Expressiveness indexChat & Writing52.433.4−19.2
FrontierMath Tiers 1-3 (v2) - PrivateMath71.289.8+18.6
SimpleQA Verified (Epoch anti-abstention)General QA42.560.7+18.2
NoCoT-Bench NCRI 15.2General QA81.6115+18.1
ZeroBench (Main questions, pass@5)Multimodal23.041.0+18.0
ARC-AGI-3 (Semi-Private)General QA5.823.0+17.2
Terminal-Bench-Science 0.1Science12.930.0+17.1
ZeroBench Main Mean pass@1 (five samples)Multimodal11.828.8+17.0
Legal Research BenchLegal45.228.8−16.3
GBENCH One-shot Coding GScoreGames53.870.1+16.3
EnterpriseBench: CoreCraftProfessional Work58.543.1−15.4
Harvey LAB (Mercor run, undisclosed ~213-task cohort)Legal18.84.1−14.7
Code MigrationCode44.257.2+13.0
Terminal-Bench 4.0Code31.844.4+12.6
Bug Hunt BenchCode40.227.9−12.3
NoCoT-Bench NCKI (kspine v3)General QA102119+11.9
Artificial Analysis Intelligence Index v5.0.1Indexes32.644.2+11.6
ARC-AGI-2General QA78.189.6+11.5
AA-Omniscience IndexIndexes4.727.1+11.2
VCT (SecureBio)Science42.954.0+11.1
WCB (SecureBio)Science49.660.5+10.9
VCT v2 (SecureBio)Science38.549.3+10.8
DeepsecBench — Revalidated V1 (233 issues)Code30.040.7+10.8
APEX-Agents 1.1Professional Work79.468.7−10.7
WeirdML v3Science9.219.7+10.5
Terminal-Bench 4.0 (Artificial Analysis)Code33.843.9+10.1
Chess Puzzles (task version 2.0)Games46.056.0+10.0
Excel Modeling BenchmarkFinance & Accounting61.871.5+9.7
GameBench 2Games69.979.4+9.5
LMArena Agent Arena Net Improvement (Steering Burden)Agents2.06.9+9.3
Bio Capabilities Index (SecureBio)Indexes149161+9.1
HealthBench Professional (Length-Adjusted)Health & Medicine51.760.8+9.1
Wiz Cyber Model Arena v3.9Cybersecurity60.369.3+9.0
AA-Briefcase v1.1 EloAgents14161266−8.2
GDP.pdf (Artificial Analysis, All-pass)Professional Work17.225.2+8.0
Finance Agent v2Finance & Accounting57.049.0−7.9
AA-PersonalityBench Imagination indexChat & Writing37.229.6−7.7
Harvey's Legal Agent BenchmarkLegal9.21.7−7.5
HPCT (SecureBio)Science60.567.8+7.3
Medical Long Context Reasoning (MLCR-AA, October 2026 methodology)Health & Medicine26.733.9+7.2
MBCT (SecureBio)Science53.961.1+7.2
FrontierCode 2.0 (Artificial Analysis)Code28.535.6+7.1
Ruby LLM Benchmarks — Program FixerCode72.079.0+7.1
Mystery Game PuzzlesGames49.056.0+7.0
DAYJOB: Finance (October 2026 rescoring)Finance & Accounting12.519.5+7.0
CritPt (avg@5)Science24.030.9+6.9
AutomationBench-AA 1.0.6Agents68.561.7−6.8
DualEntry Accounting AI BenchmarkFinance & Accounting82.276.2−6.0
CWE-Bench-AA (Artificial Analysis)Cybersecurity69.264.2−5.0
APEX-1 (Mercor, mean score)Professional Work68.363.6−4.7
LMArena Agent Arena Praise vs ComplaintAgents1.210.0+4.4
LMArena Agent Arena Bash ReliabilityAgents2.410.9+4.3
APEX-SWECode49.145.0−4.1
Epiq LAER CorporateBenchProfessional Work75.279.0+3.8
Epoch Capabilities Index (ECI)Indexes159164+3.8
LAB-Bench CloningScenarios (SecureBio)Science94.898.5+3.7
Artificial Analysis Progress Index v1Indexes324334+3.6
LMArena Agent Arena Steering BurdenAgents4.010.1+3.1
Humanity's Last Exam (Text-Only)General QA43.346.3+3.0
MMMU ProMultimodal79.782.5+2.8
LAB-Bench LitQA2 (SecureBio)Science49.051.8+2.8
LiveBench 2.0Agents79.281.9+2.7
Artificial Analysis Intelligence Index v4.3.2Indexes45.147.6+2.5
Bedside BenchHealth & Medicine85.788.1+2.4
DeepSWE v1.1 (Fireworks, avg@3)Code69.667.3−2.4
WMDP-Bio (SecureBio)Science87.990.0+2.1
SciCode 1.0.1 (Artificial Analysis)Code55.957.6+1.7
τ³-Banking (Mercor run, bm25_grep retrieval)Agents41.643.0+1.4
DAYJOB: Healthcare (October 2026 rescoring)Health & Medicine6.07.2+1.2
GPQA-Bio (SecureBio)Science87.888.9+1.1
Humanity's Sixth SenseMultimodal30.131.2+1.1
LMArena Agent Arena Confirmed SuccessAgents2.34.5+1.1
AA-LCR v1.1Search82.783.7+1.0
ARC-AGI-1 Semi-PrivateGeneral QA94.595.5+1.0
Vals Index v2.1Indexes56.657.5+1.0
PubMedQA (SecureBio)Health & Medicine80.080.9+0.9
LAB-Bench ProtocolQA (SecureBio)Science81.981.1−0.8
Design Arena Overall Code CategoriesCode12891293+0.7
OTIS Mock AIME 2024-2025Math99.4100.0+0.6
APEX-Accounting 1.1Finance & Accounting59.359.7+0.4
LMArena Agent Arena Tool HallucinationAgents0.31.0+0.4
τ³-Banking (Fireworks, banking_knowledge)Agents46.847.0+0.2
MMLU-Bio (SecureBio)Science90.790.8+0.1
Vibe Code Bench v1.1Code87.787.8+0.1

DeepSeek V4.1 Flash 1017vsGPT-6 Solnormalised score marginsData Nov 15, 2026Data: Artificial Analysis, SecureBio, Vals AI and 27 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…