Atlas

Compare Models

DeepSeek V4.1 Flash 1017 vs Grok 4.7

DeepSeek V4.1 Flash 1017 leads by 0.3 capability points (≈95% interval −2.0 to +2.7): statistically tied · P(higher) ≈ 61%

Wins 74 of 168 shared benchmarks (73 lost, 21 tied)

Capability vs. relative cost
DeepSeek V4.1 Flash 1017Grok 4.7Closed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

DeepSeek V4.1 Flash 1017vsGrok 4.7capability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins74 – 73 · 168 shared, 21 tied

Comparing matching setups only (5 benchmarks excluded) · suites counted once
← DeepSeek V4.1 Flash 1017Grok 4.7 →

DeepSeek V4.1 Flash 1017vsGrok 4.7wins by categoryData Nov 15, 2026Data: Artificial Analysis, Vals AI, SecureBio and 39 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Grok 4.7 minus DeepSeek V4.1 Flash 1017.

AA-PersonalityBench Organization indexChat & Writing88.629.6−59.6
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.040.0+40.0
AA-PersonalityBench Informativeness indexChat & Writing71.433.4−38.5
WARC-Bench (192-task test split)Agents70.535.6−34.9
HardGeoBench Avg DistanceMultimodal32012104+33.9
LAB-Bench LitQA2 (SecureBio)Science49.016.9−32.1
AA-PersonalityBench Slop indexChat & Writing39.18.6−30.8
LAB-Bench ProtocolQA (SecureBio)Science81.951.4−30.5
FrontierMath Tier 4 (v2) - PrivateMath46.317.1−29.3
AA-PersonalityBench Transparency indexChat & Writing86.758.1−28.8
HardGeoBench Country AccuracyMultimodal15.043.8+28.8
AA-PersonalityBench Expressiveness indexChat & Writing52.475.3+23.1
BrokenArXiv 09/2026 (Native Harnesses)Math49.727.7−22.0
DAYJOB: Healthcare (October 2026 rescoring)Health & Medicine6.027.6+21.6
DAYJOB: LawLegal6.227.6+21.4
HardGeoBench <=100miMultimodal7.527.5+20.0
Mystery Game PuzzlesGames49.029.0−20.0
HANDBOOK.md Agents v1.1Professional Work24.044.0+20.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - ImmigrationLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal20.00.0−20.0
RiemannBench (November 2026 rescoring)Math38.457.6+19.2
DeepsecBench — Revalidated V1 (233 issues)Code30.010.9−19.1
CyberOffenseCybersecurity72.854.4−18.4
FrontierMath Tiers 1-3 (v2) - PrivateMath71.253.0−18.2
AA-PersonalityBench Warmth indexChat & Writing27.744.8+17.3
ARC-AGI-2General QA78.161.4−16.7
DAXBench v2Code77.792.6+14.9
LAB-Bench CloningScenarios (SecureBio)Science94.880.1−14.7
Code Migration v1.1Code67.553.2−14.3
DAYJOB: Finance (October 2026 rescoring)Finance & Accounting12.526.5+14.0
FoodTruck BenchGames5328942561−13.8
HardGeoBench <=10miMultimodal3.817.5+13.8
AA-Omniscience IndexIndexes4.732.0+13.7
SimpleQA Verified (Epoch anti-abstention)General QA42.556.0+13.5
Bug Hunt BenchCode40.727.6−13.1
FrontierML ($200 wallet)Agents23.510.9−12.6
DAYJOB: AccountingFinance & Accounting7.620.0+12.4
AA-Briefcase v1.1 EloAgents14161629+11.7
BrokenArXiv 08/2026 (Native Harnesses)Math46.735.1−11.6
AA-PersonalityBench Conviction indexChat & Writing94.382.9−11.5
APEX-Agents 1.1Professional Work79.468.7−10.7
EnterpriseBench: CoreCraftProfessional Work58.568.2+9.7
Artificial Analysis Finance & Accounting IndexIndexes42.251.7+9.5
Artificial Analysis Legal IndexIndexes37.747.2+9.5
FrontierCode 2.0 - Pass RateCode23.032.3+9.3
τ³-Banking (Mercor run, bm25_grep retrieval)Agents41.632.6−9.0
HoneyBench v0.2 (hack rate)Safety76.485.1−8.7
FrontierCode 2.0 - ScoreCode20.228.9+8.7
VulcanBench Frontier v4 (functional pass@1)Code91.3100.0+8.7
WCB (SecureBio)Science49.657.6+8.0
MBCT (SecureBio)Science53.946.1−7.8
ATLAS (Exa, 200-task subset)Search31.939.5+7.6
Artificial Analysis Economics IndexIndexes38.545.7+7.2
HPCT (SecureBio)Science60.553.7−6.8
Artificial Analysis Intelligence Index v5.0Indexes28.535.3+6.8
ALE-BenchCode20121823−6.7
Terminal-Bench 4.0Code31.037.6+6.6
CritPt (avg@5)Science24.018.0−6.0
DeepSWE v1.1Code77.071.0−6.0
GDP.pdf (Artificial Analysis, All-pass)Professional Work17.223.2+6.0
Tax Agent BenchFinance & Accounting74.969.0−5.9
DualEntry Accounting AI BenchmarkFinance & Accounting82.288.1+5.9
Bio Capabilities Index (SecureBio)Indexes149141−5.7
Artificial Analysis Strategy & Ops IndexIndexes49.254.8+5.6
Medical Long Context Reasoning (MLCR-AA, October 2026 methodology)Health & Medicine26.721.1−5.6
GPQA-Bio (SecureBio)Science87.882.5−5.3
Artificial Analysis Healthcare & Medical IndexIndexes41.346.6+5.3
Excel Modeling BenchmarkFinance & Accounting61.867.0+5.2
LiveBench 2.0Agents79.274.1−5.1
Chess Puzzles (task version 2.0)Games46.041.0−5.0
Artificial Analysis Coding Agent Index v2.1Indexes36.841.6+4.8
Finance Agent v2Finance & Accounting57.052.3−4.7
RedlineBenchLegal44.949.5+4.6
Arena Image-to-WebDevCode16091589−4.4
ARC-AGI-1 Semi-PrivateGeneral QA94.590.2−4.3
APEX-SWECode49.153.4+4.3
ARC-AGI-3 (Semi-Private)General QA5.810.0+4.2
Artificial Analysis Engineering IndexIndexes37.641.8+4.2
HLAB-Vals-1.1Legal14.218.3+4.2
FrontierCode 1.1 Main (Artificial Analysis)Code43.739.6−4.1
Terminal-Bench 4.0 (Artificial Analysis)Code28.824.7−4.0
APEX-Accounting 1.1Finance & Accounting59.355.3−4.0
Ruby LLM Benchmarks — Program FixerCode72.068.0−4.0
DAYJOB: TaxFinance & Accounting1.25.0+3.8
VCT v2 (SecureBio)Science38.542.0+3.5
LMArena Agent Arena Bash ReliabilityAgents2.49.3+3.5
CritPt v2 (agentic, avg@5)Science52.355.7+3.4
DAYJOB: Data ScienceProfessional Work3.36.7+3.4
LMArena Agent Arena Steering BurdenAgents4.0-2.3−3.2
Design Arena Overall Code CategoriesCode12891270−3.2
Concept Synth INDUCTION Challenge100 Round 1General QA21.024.0+3.0
LMArena Code Arena WebDevCode16641641−3.0
AutomationBench-AA 1.0.6Agents68.565.6−2.9
CWE-bench v2 Pass@1Cybersecurity57.654.7−2.9
APEX-1 (Mercor, mean score)Professional Work68.365.4−2.9
Artificial Analysis Intelligence Index v5.0.1Indexes32.635.5+2.9
Humanity's Last Exam DiamondGeneral QA20.823.6+2.8
AA-LCR v1.1Search82.780.0−2.7
Epoch Capabilities Index (ECI)Indexes159156−2.7
Vals Index v2.2Indexes61.358.6−2.6
ArXivMath 08/2026 (Native Harnesses)Math43.946.5+2.6
GBENCH One-shot Coding GScoreGames53.856.3+2.5
VIALS-AA (pass^3)Multimodal40.437.9−2.5
GameBench 2Games69.972.3+2.4
Finance Agent v2.1Finance & Accounting62.960.6−2.3
Excel Modeling Benchmark v1.1Finance & Accounting69.271.5+2.3
CyberGym-E2E-AA (Artificial Analysis)Cybersecurity76.374.0−2.3
Visa Agentic Commerce BenchmarkAgents57.860.0+2.2
BuseyBench SVGMultimodal4.34.6+2.2
Vals Finance IndexIndexes60.958.8−2.1
ReactBench (39 tasks)Code34.432.3−2.1
DeepsecBench-AA (Artificial Analysis)Cybersecurity24.926.9+2.0
AA-PersonalityBench Imagination indexChat & Writing37.235.3−1.9
Legal Research BenchLegal45.247.1+1.9
SciCode 1.0.1 (Artificial Analysis)Code55.957.8+1.9
Vals Index v2.1Indexes56.654.9−1.6
MKA (Artificial Analysis)Professional Work38.540.1+1.6
Concept Synth INDUCTION Challenge64 Round 1General QA32.834.4+1.6
o11y-bench 1.1 Pass^3Agents75.076.6+1.6
MMMU ProMultimodal79.778.2−1.6
Vibe Code Bench v1.1Code87.786.2−1.6
GDPxProfessional Work5.43.9−1.5
CursorBench 4.0Code47.746.3−1.4
OTIS Mock AIME 2024-2025Math99.498.1−1.4
LMArena Agent Arena Confirmed SuccessAgents2.35.1+1.4
Artificial Analysis Intelligence Index v4.3.2Indexes45.146.4+1.4
LMArena Agent Arena Net Improvement (Steering Burden)Agents2.02.6+1.0
Terminal-Bench-Science 0.1Science10.59.5−1.0
LMArena Agent Arena Praise vs ComplaintAgents1.23.1+0.9
WMDP-Bio (SecureBio)Science87.988.8+0.9
CWE-Bench-AA (Artificial Analysis)Cybersecurity69.268.3−0.8
VIALS-AA (average of 3)Multimodal53.252.4−0.8
Vals Legal IndexIndexes71.972.7+0.8
PubMedQA (SecureBio)Health & Medicine80.079.2−0.8
Harvey HLAB — Criteria Pass RateLegal89.790.5+0.8
WeirdML v3Science9.28.5−0.7
Harvey LAB (Mercor run, undisclosed ~213-task cohort)Legal18.819.5+0.7
VIALS-AA (pass@3)Multimodal66.565.8−0.6
Code MigrationCode44.244.8+0.6
FrontierPhysics v0.1Science3.64.2+0.6
Access to Justice BenchmarkLegal73.173.7+0.6
Artificial Analysis Progress Index v1Indexes324323−0.4
LMArena Agent Arena Tool HallucinationAgents0.31.0+0.4
MMLU-Bio (SecureBio)Science90.790.9+0.2
Humanity's Last Exam (Text-Only)General QA43.343.1−0.2
Aikido Known-CVE Detection (32 CVEs) single-run averageCybersecurity70.870.80.0
ArXivMath 09/2026 (Native Harnesses)Math45.345.30.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal20.020.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal20.020.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal20.020.00.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal20.020.00.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal20.020.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal20.020.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0

DeepSeek V4.1 Flash 1017vsGrok 4.7normalised score marginsData Nov 15, 2026Data: Artificial Analysis, Vals AI, SecureBio and 39 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…