Atlas

Compare Models

DeepSeek V4.1 Flash 1017 vs GPT-6 Astra

GPT-6 Astra leads by 6.3 capability points (≈95% interval +3.7 to +8.9): a clear lead over DeepSeek V4.1 Flash 1017 · P(higher) ≈ 100%

Wins 112 of 171 shared benchmarks (42 lost, 17 tied)

Capability vs. relative cost
DeepSeek V4.1 Flash 1017GPT-6 AstraClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

DeepSeek V4.1 Flash 1017vsGPT-6 Astracapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins42 – 112 · 171 shared, 17 tied

Comparing matching setups only (85 benchmarks excluded) · suites counted once
← DeepSeek V4.1 Flash 1017GPT-6 Astra →

DeepSeek V4.1 Flash 1017vsGPT-6 Astrawins by categoryData Nov 15, 2026Data: Vals AI, Artificial Analysis, SecureBio and 36 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6 Astra minus DeepSeek V4.1 Flash 1017.

ARC-AGI-3 (Semi-Private)General QA5.898.6+92.7
CyberGym-E2E-AA (Artificial Analysis)Cybersecurity76.30.0−76.3
Concept Synth INDUCTION Challenge100 Round 1General QA21.094.0+73.0
HardGeoBench Avg DistanceMultimodal3201908+70.8
Concept Synth INDUCTION Challenge64 Round 1General QA32.898.4+65.6
FrontierMath Tier 4 (v2) - PrivateMath46.397.6+51.3
AA-PersonalityBench Conviction indexChat & Writing94.344.8−50.0
LisanBench Difficulty-WeightedGames77333113−49.5
Terminal-Bench-Science 0.1Science10.559.5+49.0
AA-PersonalityBench Warmth indexChat & Writing27.775.3+48.1
HardGeoBench Country AccuracyMultimodal15.060.0+45.0
HardGeoBench <=100miMultimodal7.551.3+43.7
RiemannBench (November 2026 rescoring)Math38.481.6+43.2
Humanity's Last Exam DiamondGeneral QA20.863.8+43.0
LisanBench Path LengthGames211077393−42.6
NoCoT-Bench NCRI 15.2General QA81.6159+42.5
AA-PersonalityBench Informativeness indexChat & Writing71.429.6−42.3
AA-PersonalityBench Imagination indexChat & Writing37.277.2+40.4
ALE-BenchCode20123128+39.5
Mystery Game PuzzlesGames49.084.0+35.0
HardGeoBench <=10miMultimodal3.837.5+33.8
SimpleQA Verified (Epoch anti-abstention)General QA42.575.6+33.1
WeirdML v3Science9.242.2+33.0
CritPt v2 (agentic, avg@5)Science52.383.7+31.4
ZeroBench Main Mean pass@1 (five samples)Multimodal11.843.2+31.4
DAYJOB: LawLegal6.236.9+30.7
BuseyBench SVGMultimodal4.37.3+29.7
ZeroBench (Main questions, pass@5)Multimodal23.052.0+29.0
Chess Puzzles (task version 2.0)Games46.074.0+28.0
Terminal-Bench 4.0Code31.859.6+27.8
Artificial Analysis Intelligence Index v5.0Indexes28.555.3+26.7
HoneyBench v0.2 (hack rate)Safety76.450.8+25.6
GBENCH One-shot Coding GScoreGames53.879.2+25.5
Terminal-Bench 4.0 (Artificial Analysis)Code28.854.0+25.3
DAYJOB: Finance (October 2026 rescoring)Finance & Accounting12.537.3+24.8
GameBench 2Games69.994.3+24.5
Artificial Analysis Intelligence Index v5.0.1Indexes32.656.9+24.3
Artificial Analysis Engineering IndexIndexes37.661.8+24.2
LMArena Agent Arena Net Improvement (Steering Burden)Agents2.014.5+23.7
Code MigrationCode44.267.7+23.5
Humanity's Sixth SenseMultimodal30.153.6+23.5
AA-PersonalityBench Slop indexChat & Writing39.116.2−23.1
FrontierMath Tiers 1-3 (v2) - PrivateMath71.293.7+22.5
DAYJOB: AccountingFinance & Accounting7.630.0+22.4
DAYJOB: Healthcare (October 2026 rescoring)Health & Medicine6.028.4+22.4
VIALS-AA (pass^3)Multimodal40.462.7+22.4
NoCoT-Bench NCKI (kspine v3)General QA102131+20.4
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - ImmigrationLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal20.00.0−20.0
AA-Omniscience IndexIndexes4.743.7+19.5
VCT v2 (SecureBio)Science38.558.0+19.5
AA-PersonalityBench Organization indexChat & Writing88.669.5−19.2
DAXBench v2Code77.796.8+19.1
Artificial Analysis Economics IndexIndexes38.557.2+18.7
WARC-Bench (192-task test split)Agents70.589.1+18.6
DAYJOB: Data ScienceProfessional Work3.321.3+18.0
RedlineBenchLegal44.962.8+17.9
Artificial Analysis Legal IndexIndexes37.755.6+17.8
ARC-AGI-2General QA78.195.0+16.9
EnterpriseBench: CoreCraftProfessional Work58.542.6−15.9
VCT (SecureBio)Science42.958.5+15.6
LMArena Agent Arena Praise vs ComplaintAgents1.231.7+15.2
Artificial Analysis Finance & Accounting IndexIndexes42.257.4+15.2
VIALS-AA (average of 3)Multimodal53.268.3+15.1
GDP.pdf (Artificial Analysis, All-pass)Professional Work17.232.2+15.0
Medical Long Context Reasoning (MLCR-AA, October 2026 methodology)Health & Medicine26.741.7+15.0
Build-A-Chip-Bench (20-task index run)Agents0.515.0+14.5
FrontierCode 2.0 (Artificial Analysis)Code28.542.4+13.9
Code Migration v1.1Code67.580.6+13.1
WCB (SecureBio)Science49.662.6+13.0
Artificial Analysis Healthcare & Medical IndexIndexes41.353.7+12.4
Humanity's Last ExamGeneral QA42.654.8+12.2
Hebbia Financial Services BenchmarkFinance & Accounting60.372.3+12.0
DeepsecBench-AA (Artificial Analysis)Cybersecurity24.936.9+12.0
MKA (Artificial Analysis)Professional Work38.550.2+11.7
Tax Agent BenchFinance & Accounting74.963.3−11.6
Bio Capabilities Index (SecureBio)Indexes149164+11.2
Finance Agent v2 - AdjustmentsFinance & Accounting58.448.3−10.1
MBCT (SecureBio)Science53.963.9+10.0
Wiz Cyber Model Arena v3.9Cybersecurity60.370.2+9.9
Excel Modeling BenchmarkFinance & Accounting61.871.7+9.9
Finance Agent v2 - ComparablesFinance & Accounting47.537.8−9.8
Harvey LAB (Mercor run, undisclosed ~213-task cohort)Legal18.89.2−9.6
Artificial Analysis Progress Index v1Indexes324350+9.1
Artificial Analysis Strategy & Ops IndexIndexes49.258.0+8.8
HPCT (SecureBio)Science60.569.0+8.5
VulnPR-100Code54.046.0−8.0
Epoch Capabilities Index (ECI)Indexes159169+7.9
o11y-bench 1.1 Pass^3Agents79.787.5+7.8
Bedside BenchHealth & Medicine85.793.4+7.8
CritPt (avg@5)Science24.031.7+7.7
AA-PersonalityBench Transparency indexChat & Writing86.794.3+7.7
DeepsecBench — Revalidated V1 (233 issues)Code30.037.6+7.7
Artificial Analysis Intelligence Index v4.3.2Indexes45.152.7+7.6
HANDBOOK.md Agents v1.1Professional Work24.031.4+7.4
LAB-Bench ProtocolQA (SecureBio)Science81.989.3+7.4
HLAB-Vals-1.1Legal14.27.1−7.1
LiveBench 2.0Agents79.286.1+6.9
VIALS-AA (pass@3)Multimodal66.573.3+6.8
Epiq LAER CorporateBenchProfessional Work75.282.0+6.8
MMMU ProMultimodal79.786.4+6.7
Visa Agentic Commerce BenchmarkAgents57.864.4+6.7
LMArena Agent Arena Confirmed SuccessAgents2.314.0+5.9
CWE-Bench-AA (Artificial Analysis)Cybersecurity69.263.3−5.8
LAB-Bench LitQA2 (SecureBio)Science49.054.8+5.8
Finance Agent v2 - Earnings AnalysisFinance & Accounting75.369.5−5.8
AA-PersonalityBench Expressiveness indexChat & Writing52.458.1+5.8
Legal Research BenchLegal45.239.4−5.8
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting74.168.4−5.7
APEX-Accounting 1.1Finance & Accounting59.364.7+5.4
LMArena Agent Arena Steering BurdenAgents4.014.8+5.4
Vals Finance IndexIndexes60.955.6−5.2
LAB-Bench CloningScenarios (SecureBio)Science94.8100.0+5.2
τ³-Banking (Mercor run, bm25_grep retrieval)Agents41.646.7+5.1
Finance Agent v2 - All PassFinance & Accounting45.240.3−5.0
AA-Briefcase v1.1 EloAgents14161503+4.8
Finance Agent v2 - Disclosure AnalysisFinance & Accounting66.361.7−4.6
APEX-Agents 1.1Professional Work79.475.1−4.3
GDPxProfessional Work5.49.6+4.2
ARC-AGI-1 Semi-PrivateGeneral QA94.598.5+4.0
Finance Agent v2 - PrecedentsFinance & Accounting26.530.3+3.8
DAYJOB: TaxFinance & Accounting1.24.8+3.6
Excel Modeling Benchmark v1.1Finance & Accounting69.272.7+3.5
Vals Legal IndexIndexes71.968.4−3.5
FrontierCode 1.1 Main (Artificial Analysis)Code43.747.0+3.3
Ruby LLM Benchmarks — Program FixerCode72.075.1+3.2
Aikido Known-CVE Detection (32 CVEs)Cybersecurity87.590.6+3.1
WMDP-Bio (SecureBio)Science87.990.8+2.9
DeepSWE v1.1Code77.074.1−2.9
Finance Agent v2 - Market AnalysisFinance & Accounting62.765.5+2.9
LMArena Agent Arena Bash ReliabilityAgents2.47.8+2.7
Bug Hunt BenchCode40.242.9+2.7
AA-LCR v1.1Search82.780.0−2.7
Finance Agent v2.1Finance & Accounting62.960.4−2.5
Harvey HLAB — Criteria Pass RateLegal89.787.5−2.3
τ³-Banking (Fireworks, banking_knowledge)Agents46.844.8−2.0
PubMedQA (SecureBio)Health & Medicine80.081.2+1.2
Finance Agent v2 - Financial ModelingFinance & Accounting22.821.6−1.2
APEX-1 (Mercor, mean score)Professional Work68.369.3+1.0
DualEntry Accounting AI BenchmarkFinance & Accounting82.281.2−1.0
Legacy-Bench (Fireworks, avg@3)Code56.355.7−0.7
MMLU-Bio (SecureBio)Science90.791.3+0.6
SciCode 1.0.1 (Artificial Analysis)Code55.956.5+0.6
OTIS Mock AIME 2024-2025Math99.4100.0+0.6
APEX-SWECode49.148.6−0.5
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting79.278.8−0.4
GPQA-Bio (SecureBio)Science87.887.4−0.4
LMArena Agent Arena Tool HallucinationAgents0.31.0+0.4
AutomationBench-AA 1.0.6Agents68.568.50.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal20.020.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal20.020.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0

DeepSeek V4.1 Flash 1017vsGPT-6 Astranormalised score marginsData Nov 15, 2026Data: Vals AI, Artificial Analysis, SecureBio and 36 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…