Atlas

Compare Models

DeepSeek V4.1 Flash vs Grok 4.7

Grok 4.7 leads by 1.2 capability points (≈95% interval −0.8 to +3.2): statistically tied · P(higher) ≈ 88%

Wins 124 of 224 shared benchmarks (81 lost, 19 tied)

Capability vs. relative cost
DeepSeek V4.1 FlashGrok 4.7Closed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

DeepSeek V4.1 FlashvsGrok 4.7capability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins81 – 124 · 224 shared, 19 tied

Comparing matching setups only (27 benchmarks excluded) · suites counted once
← DeepSeek V4.1 FlashGrok 4.7 →

DeepSeek V4.1 FlashvsGrok 4.7wins by categoryData Nov 15, 2026Data: Vals AI, Artificial Analysis, Arena and 46 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Grok 4.7 minus DeepSeek V4.1 Flash.

RSI Arena Stage 1 human ratingAgents8871038+72.6
AA-PersonalityBench Organization indexChat & Writing10029.6−71.2
Design Creativity BenchCode43.662.2+59.2
CyberGym-E2E-AA (Artificial Analysis)Cybersecurity22.974.0+51.1
AA-PersonalityBench Informativeness indexChat & Writing79.133.4−46.2
SheetBench-50Finance & Accounting58.414.3−44.1
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.040.0+40.0
CloudOpsBench v1.0Agents21.856.5+34.7
KernelBench Mega Normalized (RTX PRO 6000)Code48.218.0−30.2
AA-PersonalityBench Slop indexChat & Writing37.28.6−28.8
ProofBench v1.1Math54.026.0−28.0
SWE-bench Verified Extended (Mercor held-out)Code82.755.3−27.4
DistressBenchSafety79.852.4−27.4
BrokenArXiv 09/2026 (Native Harnesses)Math54.727.7−27.0
EnterpriseBench: CoreCraftProfessional Work41.568.2+26.7
Terminal-Bench 2.1 Extended (Mercor held-out)Code38.411.8−26.6
Creative Writing v3 EloChat & Writing15402012+23.9
SWE Atlas Codebase QnA Extended (Mercor held-out)Code36.715.3−21.4
AA-AnalystAgent (pass^5)Professional Work35.055.0+20.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal20.00.0−20.0
BrokenArXiv 08/2026 (Native Harnesses)Math54.835.1−19.6
AA-Omniscience IndexIndexes-5.332.0+18.7
APEX-Agents 1.1Professional Work50.568.7+18.2
SAFE-Teen 106 worst branch: Health Advice (under-18 prompt)Safety37.019.2+17.8
IOI (Vals v2)Code40.357.7+17.4
Concept Synth INDUCTION Challenge64 Round 1General QA17.234.4+17.2
DAXBench v2Code75.592.6+17.0
Artificial Analysis Cyber Index v1Indexes40.856.4+15.6
SAFE-Teen 106 worst branch: Relationships (under-18 prompt)Safety19.85.1+14.7
FrontierMath Tiers 1-3 (v2) - PrivateMath67.453.0−14.4
Mystery Game PuzzlesGames43.029.0−14.0
SWE-bench MultilingualCode98.284.2−14.0
AA-PersonalityBench Conviction indexChat & Writing96.282.9−13.5
AA-PersonalityBench Expressiveness indexChat & Writing61.975.3+13.5
BrokenArXiv 06/2026Math39.452.8+13.4
Furniture Assembly BenchmarkMultimodal34.220.8−13.3
FrontierCUA pass@1Agents6.720.0+13.3
AA-Briefcase v1.1 EloAgents14151657+13.3
SimpleQA Verified (Epoch anti-abstention)General QA42.956.0+13.1
SAFE-Teen 106 worst branch: Urgent Safety (no prompt)Safety31.444.4−13.0
Concept Synth INDUCTION Challenge100 Round 1General QA11.024.0+13.0
CWE-bench v1 Programmatic Pass@1Cybersecurity55.068.0+13.0
Extended NYT Connections 940 (Quadratic v1)Games89.676.8−12.8
ArXivMath 06/2026Math57.670.1+12.5
CadQueryEval (Corrected Geometry)Code76.088.0+12.0
Eval Connections One-Shot with TrapsGames73.085.0+12.0
SAFE-Teen 106: Health Advice (under-18 prompt)Safety23.612.0+11.6
ARC-AGI-2General QA72.961.4−11.5
BrokenArXiv 05/2026Math28.540.0+11.5
SAFE-Teen 106: Health Advice (no prompt)Safety30.218.7+11.5
LLM Chess EloGames1087822−10.9
LLM Creative Story WritingChat & Writing-0.50.4+10.4
MetagenomicsBench V0Science38.048.3+10.3
Excel Modeling BenchmarkFinance & Accounting57.267.0+9.8
FoodTruck BenchGames5017942561−9.8
FrontierMath Tier 4 (v2) - PrivateMath26.817.1−9.8
Creative Writing v3 Rubric ScoreChat & Writing14.115.9+9.5
CharXiv Extended (Mercor held-out)Multimodal68.978.2+9.3
Longform Creative WritingChat & Writing74.083.0+9.0
MedCodeHealth & Medicine41.249.6+8.4
GBENCH One-shot Coding GScoreGames48.056.3+8.3
Arena Alignment IndexSafety74.882.7+7.9
Medical Long Context Reasoning (MLCR-AA)Health & Medicine22.815.0−7.8
Harvey LAB-AA v1.1 (hallucination-gated)Legal1.79.4+7.8
LMArena Agent Arena Chat Net Improvement (Steering Burden)Agents1.64.8+7.7
HealthBenchHealth & Medicine61.954.2−7.7
Design Arena Overall Code CategoriesCode13161270−7.6
Terminal-Bench Science 0.1 (Vals)Science4.311.4+7.1
SAGEMath47.940.8−7.1
Chess Puzzles (task version 2.0)Games48.041.0−7.0
Artificial Analysis Intelligence Index v4.3.2Indexes39.546.4+7.0
Bug Hunt BenchCode20.627.6+7.0
ARC-AGI-3 (Semi-Private)General QA3.210.0+6.9
Tax Agent BenchFinance & Accounting62.569.0+6.5
SWE Atlas - Codebase QnACode53.547.0−6.5
Conceptual Reasoning IndexIndexes58.463.1+6.4
Terminal-Bench 4.0Code31.237.6+6.4
DTBenchGeneral QA89.996.0+6.1
ACCoRDGeneral QA68.174.0+6.0
Code2Video-AA EloCode10531127+6.0
SciCode 1.0.1 (Artificial Analysis)Code51.957.8+5.9
Legal Research BenchLegal41.347.1+5.8
LMArena Text Arena EloChat & Writing14751442−5.8
Code Migration v1.1Code58.953.2−5.7
VIALSMultimodal12.618.2+5.6
Terminal-Bench 2.1Code90.685.0−5.6
MedXpertQA MM (200-question set)Health & Medicine63.569.0+5.5
OpenWorldEngineer (Mean@8)Code12.47.0−5.3
LLM Debate BenchmarkChat & Writing15821620+5.3
Aikido Known-CVE Detection (32 CVEs) single-run averageCybersecurity65.670.8+5.2
GDPval-AA v2.1Professional Work16001718+5.2
τ³-Banking (Mercor run, bm25_grep retrieval)Agents37.832.6−5.2
Vals Science IndexIndexes29.634.8+5.2
LMArena Agent Arena Chat Bash ReliabilityAgents-1.09.1+5.1
HealthBench (length-adjusted, Artificial Analysis)Health & Medicine60.956.0−4.9
Finance Agent v2.1Finance & Accounting65.460.6−4.8
LMArena Agent Arena Code Bash ReliabilityAgents-0.29.5+4.8
ITBench-AAAgents46.942.1−4.8
BuseyBench SVGMultimodal4.14.6+4.8
ArXivMath 09/2026 (Native Harnesses)Math40.645.3+4.7
Arena Image-to-WebDevCode16101589−4.7
Senior SWE-BenchCode22.727.4+4.7
LMArena Agent Arena Work Bash ReliabilityAgents0.19.4+4.6
Humanity's Last Exam DiamondGeneral QA19.123.6+4.5
SAFE-Teen 106: Relationships (no prompt)Safety27.131.5−4.4
ARC-AGI-1 Semi-PrivateGeneral QA94.590.2−4.3
DeepsecBench-AA (Artificial Analysis)Cybersecurity31.226.9−4.3
CyberBench v1.1 OverallCybersecurity73.769.5−4.2
DeepResearch Bench II (Mercor 130-task set)Search53.249.0−4.2
MysteryMechanismScience21.225.2+4.1
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine14.718.7+4.0
TaxCalcBench TY25Finance & Accounting4.08.0+4.0
LiveBench 2.0Agents78.074.1−4.0
DualEntry Accounting AI BenchmarkFinance & Accounting84.288.1+3.9
GameBench 2Games68.472.3+3.9
MedScribeHealth & Medicine85.589.4+3.9
AA-PersonalityBench Warmth indexChat & Writing41.044.8+3.8
AA-PersonalityBench Imagination indexChat & Writing31.535.3+3.8
SWE-bench VerifiedCode82.979.1−3.8
AdvancedIFChat & Writing84.080.2−3.8
SAFE-Teen 106: Urgent Safety (no prompt)Safety11.315.1−3.8
Excel Modeling Benchmark v1.1Finance & Accounting67.871.5+3.7
CritPt (avg@5)Science14.318.0+3.7
LMArena Agent Arena Chat Confirmed SuccessAgents4.912.1+3.6
BrowseComp (Mercor 130-question subset)Search79.475.8−3.6
MedXpertQA MM Extended (Mercor held-out)Health & Medicine48.452.0+3.6
Vision Arena OverallMultimodal12631249−3.6
SWE-bench Multimodal (Vals v1, 480 Tasks)Code46.542.9−3.5
AutomationBench-AA 1.0.6Agents68.965.6−3.3
SAGE v1.1Math85.488.7+3.3
Ramp Accounting BenchFinance & Accounting40.844.1+3.3
SAFE-Teen 106 worst branch: Relationships (no prompt)Safety51.147.9+3.2
FrontierCode 1.1 Extended - Pass RateCode62.065.2+3.2
DeepSWE v1.1Code74.271.0−3.2
SAFE-Teen 106: Relationships (under-18 prompt)Safety4.21.0+3.2
ChartographyMultimodal11.614.7+3.1
SRE Bench — Fully SolvedCode0.83.8+3.1
GDP.pdfProfessional Work19.822.8+3.0
Harvey HLAB — Criteria Pass RateLegal87.590.5+3.0
FrontierCode 1.1 Extended - ScoreCode56.659.4+2.7
LMArena Code Arena WebDevCode16211641+2.7
ArXivMath 08/2026 (Native Harnesses)Math43.946.5+2.6
WeirdML v3Science5.88.5+2.6
AA-LCR Extended (Mercor held-out)Search70.868.2−2.6
ArXivMath 05/2026Math58.360.8+2.5
Immigration BenchLegal65.668.0+2.4
LMCAGeneral QA47.049.4+2.4
Terminal-Bench-Science 0.1Science9.011.4+2.4
Vals Index v2Indexes57.960.2+2.4
SAFE-Teen 106: Urgent Safety (under-18 prompt)Safety7.910.1−2.2
SAFE-Teen 106 worst branch: Health Advice (no prompt)Safety41.139.2+1.9
AA-PersonalityBench Transparency indexChat & Writing60.058.1−1.9
CharXiv (descriptive and reasoning combined)Multimodal90.192.0+1.9
LMArena Agent Arena SteerabilityAgents-3.30.5+1.9
KernelBench CUDA Normalized (RTX PRO 6000)Code47.045.2−1.8
Vals Finance IndexIndexes57.158.8+1.8
AIME 2025Math98.3100.0+1.7
LMArena Agent Arena Code Net Improvement (Steering Burden)Agents2.73.5+1.7
MineBench — Bradley-Terry RatingGames18221848+1.6
APEX-SWECode51.953.4+1.5
BioMysteryBench (Vals Terminus 2)Science67.869.3+1.5
Vibe Code Bench v1.1Code84.786.2+1.4
APEX-AccountingFinance & Accounting48.146.7−1.4
Public Benefits Bench v1.1Professional Work64.365.6+1.4
Financial Coaching BenchFinance & Accounting72.874.1+1.2
Finance Agent v2Finance & Accounting53.552.3−1.2
LegalBenchLegal83.384.4+1.1
Westworld Finance Diligence BenchFinance & Accounting48.347.2−1.1
LMArena Agent Arena Work Steering BurdenAgents1.5-0.6−1.1
Terminal-Bench 4.0 (Artificial Analysis)Code26.825.8−1.0
GeneBench-Pro Public Release SubsetScience11.010.0−1.0
SpeciEval (Pooled English Runs)Safety97.496.4−1.0
SpeciEval (Refusal-Aware Retries, Historical)Safety97.496.4−1.0
LMArena Agent Arena Work Confirmed SuccessAgents5.73.8−0.9
LMArena Agent Arena Code Steering BurdenAgents2.30.4−0.9
Artificial Analysis Progress Index v1Indexes320323+0.9
RCM-Bench-1Health & Medicine89.088.2−0.8
Access to Justice BenchmarkLegal72.973.7+0.8
Code MigrationCode45.644.8−0.8
ComplexConstraintsChat & Writing49.748.9−0.8
dfbench Detection RecallCybersecurity57.356.6−0.7
MMLU ProGeneral QA88.288.8+0.6
LMArena Agent Arena Net ImprovementAgents3.94.0+0.5
LMArena Agent Arena Chat Steering BurdenAgents-0.50.6+0.5
SAFE-Teen 106 worst branch: Urgent Safety (under-18 prompt)Safety27.226.7+0.5
LMArena Agent Arena Chat Praise vs ComplaintAgents3.13.9+0.4
LMArena Agent Arena Work Praise vs ComplaintAgents3.72.8−0.4
LMArena Agent Arena Bash RecoveryAgents7.97.2−0.3
Epoch Capabilities Index (ECI)Indexes156156−0.3
YC-BenchGames16572361666389+0.3
OTIS Mock AIME 2024-2025Math98.398.1−0.3
RuneBenchGames5.75.7−0.2
LMArena Agent Arena Code Praise vs ComplaintAgents3.03.4+0.2
ALE-BenchCode18291823−0.2
APEX-1 (Mercor, mean score)Professional Work65.565.4−0.1
LMArena Agent Arena Code Confirmed SuccessAgents6.66.4−0.1
LMArena Agent Arena Code Tool HallucinationAgents0.50.7+0.1
LMArena Agent Arena Work Tool HallucinationAgents0.50.7+0.1
LMArena Agent Arena Chat Tool HallucinationAgents0.30.40.0
LMArena Agent Arena Work Net Improvement (Steering Burden)Agents2.52.50.0
CWE-Bench-AA (Artificial Analysis)Cybersecurity68.368.30.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal20.020.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal20.020.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal20.020.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
ProgramBenchCode0.50.50.0

DeepSeek V4.1 FlashvsGrok 4.7normalised score marginsData Nov 15, 2026Data: Vals AI, Artificial Analysis, Arena and 46 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…