Atlas

Compare Models

DeepSeek V4.1 Flash vs Gemini 3.7 Flash

DeepSeek V4.1 Flash leads by 0.0 capability points (≈95% interval −1.9 to +1.9): statistically tied · P(higher) ≈ 50%

Gemini 3.7 Flash wins 95 of 205 shared benchmarks (90 lost, 20 tied)

Capability vs. relative cost
DeepSeek V4.1 FlashGemini 3.7 FlashClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

DeepSeek V4.1 FlashvsGemini 3.7 Flashcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins90 – 95 · 205 shared, 20 tied

Comparing matching setups only (11 benchmarks excluded) · suites counted once
← DeepSeek V4.1 FlashGemini 3.7 Flash →

DeepSeek V4.1 FlashvsGemini 3.7 Flashwins by categoryData Nov 15, 2026Data: Vals AI, Arena, Mercor and 37 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Gemini 3.7 Flash minus DeepSeek V4.1 Flash.

SWE-bench Verified Extended (Mercor held-out)Code82.730.0−52.7
BioSecBench-Refusal V1 Clean RerunsSafety11.354.8+43.5
BabyVisionMultimodal43.373.5+30.2
CyberBench v1.1 OverallCybersecurity73.743.8−29.9
APEX-Agents 1.1Professional Work50.579.4+28.9
ChartographyMultimodal11.640.4+28.8
BrokenArXiv 06/2026Math39.411.1−28.2
IOI (Vals v2)Code40.367.8+27.6
SimpleQA Verified (Epoch anti-abstention)General QA42.969.2+26.3
BuseyBench SVGMultimodal4.16.6+25.5
AA-AnalystAgent (pass^5)Professional Work35.060.0+25.0
AA-LCR Extended (Mercor held-out)Search70.848.9−21.9
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal20.040.0+20.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal20.00.0−20.0
MedXpertQA MM (200-question set)Health & Medicine63.583.5+20.0
BrokenArXiv 05/2026Math28.58.5−20.0
P2NPCBench (NP-complete, pass@1)Code39.357.8+18.5
LLM Debate BenchmarkChat & Writing15821455−18.1
Chess Puzzles (task version 2.0)Games48.066.0+18.0
AA-Briefcase EloAgents14241116−17.9
LMArena Agent Arena Net ImprovementAgents3.9-1.3−17.0
SkillsBench without SkillsAgents61.744.9−16.8
SUPERChemScience56.772.8+16.1
AA-Omniscience IndexIndexes-5.326.5+15.9
SWE-bench MultilingualCode98.282.6−15.6
RiemannBenchMath24.039.2+15.2
LMArena Agent Arena Code Net Improvement (Steering Burden)Agents2.7-3.6−14.4
EnterpriseBench: CoreCraftProfessional Work41.555.9+14.4
Vibe Code Bench v1.1Code84.770.4−14.3
MedXpertQA MM Extended (Mercor held-out)Health & Medicine48.462.7+14.3
Excel Modeling BenchmarkFinance & Accounting57.271.3+14.1
VIALSMultimodal12.626.5+13.9
HLE Extended (Mercor held-out)General QA45.331.6−13.7
Terminal-Bench 4.0 (Artificial Analysis)Code26.813.6−13.1
LMArena Agent Arena Work Net Improvement (Steering Burden)Agents2.5-2.6−12.5
ArXivMath 04/2026Math45.057.5+12.5
MedCodeHealth & Medicine41.253.4+12.2
SpatialBench-Long V0 VerifiedScience34.922.7−12.1
Humanity's Last Exam DiamondGeneral QA19.131.2+12.1
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting69.881.8+12.0
ARC-AGI-2General QA72.984.6+11.7
MMMU-Pro Extended (Mercor held-out)Multimodal25.036.4+11.4
Finance Agent v2 - Market AnalysisFinance & Accounting63.474.5+11.2
SpatialBench V0 VerifiedScience75.464.3−11.0
TxBench Antibody Discovery V0Science39.050.0+11.0
TxBench Antibody Discovery V1 (100 evaluations)Science39.750.7+11.0
LiveBench 2.0Agents78.067.2−10.9
DistressBenchSafety79.869.0−10.8
ArXivMath 05/2026Math58.347.5−10.8
Code MigrationCode45.634.8−10.8
CharXiv Extended (Mercor held-out)Multimodal68.979.1+10.2
BioSecBench-Surveillance V1 Clean RerunsScience51.541.5−10.0
TaxCalcBench TY25Finance & Accounting4.014.0+10.0
APEX-SWECode51.942.0−9.9
DeepResearch Bench II (Mercor 130-task set)Search53.243.4−9.8
GDPval-AA v2Professional Work16321433−9.8
FrontierMath Tier 4 (v2) - PrivateMath26.836.6+9.8
Arena Code WebDev — Brand & MarketingCode16181553−9.6
Creative Writing v3 EloChat & Writing15401720+9.1
ACCoRDGeneral QA68.177.0+8.9
Wiz Cyber Model Arena v3.9Cybersecurity50.259.1+8.9
Humanity's Last ExamGeneral QA36.845.6+8.8
DeepSWE v1.1Code74.265.5−8.7
TxBench Preclinical Pharmacology V1 Clean RerunsScience46.755.3+8.7
GDPval-AA v2.1Professional Work16001404−8.6
Conceptual Reasoning IndexIndexes58.464.6+8.4
SpeciEval (Pooled English Runs)Safety97.489.0−8.3
SpeciEval (Refusal-Aware Retries, Historical)Safety97.489.0−8.3
CadQueryEval (Corrected Geometry)Code76.084.0+8.0
Senior SWE-BenchCode22.714.7−8.0
SciCode 1.0.1 (Artificial Analysis)Code51.959.8+8.0
Vision Arena OverallMultimodal12631294+7.9
Medical Long Context Reasoning (MLCR-AA)Health & Medicine22.815.0−7.8
Furniture Assembly BenchmarkMultimodal34.226.7−7.5
ComplexConstraintsChat & Writing49.742.4−7.3
BioSecBench-Function V1 Clean RerunsScience40.833.7−7.2
LMArena Agent Arena Code Confirmed SuccessAgents6.6-7.7−7.1
APEX-Accounting 1.1Finance & Accounting50.157.2+7.1
Eval Connections One-Shot with TrapsGames73.080.0+7.0
DTBenchGeneral QA89.996.8+6.9
AutomationBench-AA 1.0.6Agents68.962.0−6.9
Legal Research BenchLegal41.334.6−6.7
LMArena Agent Arena Bash RecoveryAgents7.9-5.4−6.7
Finance Agent v2 - Disclosure AnalysisFinance & Accounting61.367.7+6.4
Finance Agent v2 - All PassFinance & Accounting41.147.5+6.4
MetagenomicsBench V1Science38.031.7−6.3
MetagenomicsBench V0Science38.031.7−6.3
Arena Code WebDev — ReactCode16301584−6.2
Finance Agent v2 - Earnings AnalysisFinance & Accounting72.979.1+6.2
Mystery Game PuzzlesGames43.037.0−6.0
Terminal-Bench 2.1 Extended (Mercor held-out)Code38.444.4+6.0
Creative Writing v3 Rubric ScoreChat & Writing14.115.2+5.8
LMArena Agent Arena Work Confirmed SuccessAgents5.7-5.8−5.7
TxBench Oligo Discovery V0Science33.939.5+5.6
Finance Agent v2 - PrecedentsFinance & Accounting28.233.8+5.6
Terminal-Bench 4.0Code11.66.1−5.6
LMArena Agent Arena Code Steering BurdenAgents2.3-8.8−5.5
Agents' Last Exam - release-reported pass rate, subset unspecifiedAgents31.826.3−5.5
Finance Agent v2 - AdjustmentsFinance & Accounting49.254.7+5.5
CharXiv (descriptive and reasoning combined)Multimodal90.195.3+5.2
ArXivMath 06/2026Math57.662.5+4.9
Arena Code WebDev — FrontendCode16301592−4.8
LMArena Agent Arena Chat Bash ReliabilityAgents-1.08.6+4.8
Tax Agent BenchFinance & Accounting62.557.7−4.8
GBENCH One-shot Coding GScoreGames48.052.6+4.6
Arena Code WebDev — Content Creation ToolsCode16021574−4.4
RuneBenchGames5.76.0+4.3
ScBench V1 Clean RerunsScience54.750.4−4.3
Finance Agent v2 - ComparablesFinance & Accounting46.150.3+4.2
FrontierMath Tiers 1-3 (v2) - PrivateMath67.471.6+4.2
SpatialBench V1 Clean RerunsScience63.259.1−4.1
GDP.pdfProfessional Work19.823.8+4.0
ProofBench v1.1Math54.058.0+4.0
ScBench-Long V1 Clean RerunsScience14.818.8+4.0
LegalBenchLegal83.387.3+4.0
SkillsBench with SkillsAgents69.865.9−3.9
LMArena Agent Arena Work Steering BurdenAgents1.5-6.2−3.9
SpatialBench-Long V1 Clean RerunsScience26.229.8+3.6
LMArena Agent Arena Work Praise vs ComplaintAgents3.7-3.4−3.5
LMArena Agent Arena Chat Net Improvement (Steering Burden)Agents1.63.1+3.5
Arena Code WebDev — GamingCode16691638−3.5
Arena Code WebDev — HTMLCode16181593−3.5
EpiBench V1 Clean RerunsScience25.522.0−3.5
MineBench — Bradley-Terry RatingGames18221765−3.5
LMCAGeneral QA47.050.4+3.4
LMArena Agent Arena Code Praise vs ComplaintAgents3.0-3.5−3.2
LMArena Agent Arena Code Bash ReliabilityAgents-0.26.1+3.1
ScBench-Long V0 VerifiedScience27.324.2−3.0
LMArena Agent Arena Work Bash ReliabilityAgents0.16.1+3.0
GeneBench-Pro Public Release SubsetScience11.014.0+3.0
Terminal-Bench 2.1Code74.577.5+3.0
TxBench Oligo Discovery V1 (113 evaluations)Science34.837.8+2.9
Arena Code WebDev — Data & AnalyticsCode15601577+2.6
LMArena Text Arena EloChat & Writing14751489+2.5
LLM Creative Story WritingChat & Writing-0.5-0.7−2.5
MMLU ProGeneral QA88.290.6+2.4
Arena Image-to-WebDevCode16101600−2.2
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting67.465.3−2.1
Design Arena Overall Code CategoriesCode13161304−2.0
SWE-bench VerifiedCode82.981.0−1.9
FrontierCode 1.1 Extended - ScoreCode56.654.7−1.9
Arena Code WebDev — Consumer ProductCode15881574−1.8
ALE-BenchCode18291876+1.7
Arena Code WebDev — SimulationsCode16331618−1.7
LMArena Agent Arena SteerabilityAgents-3.3-0.0+1.6
Epoch Capabilities Index (ECI)Indexes156158+1.6
AIME 2025Math98.396.7−1.6
MedScribeHealth & Medicine85.583.9−1.6
Arena Code WebDev — FullstackCode16241629+1.5
Ramp Accounting BenchFinance & Accounting40.842.3+1.5
APEX-1 (Mercor, mean score)Professional Work65.567.0+1.5
GDPval (Mercor 213-task rubric score)Professional Work86.785.2−1.5
Vals Index v2Indexes57.959.3+1.4
FrontierCode 1.1 Extended - Pass RateCode62.060.6−1.4
Terminal-Bench Science 0.1 (Vals)Science4.35.7+1.4
BioSecBench-Function V0Science38.740.1+1.4
SAGEMath47.949.2+1.4
VoxelBench Text RatingGames18611844−1.2
OTIS Mock AIME 2024-2025Math98.397.2−1.1
Finance Agent v2 - Financial ModelingFinance & Accounting23.124.2+1.1
Harvey HLAB — Criteria Pass RateLegal87.588.6+1.1
LMArena Agent Arena Chat Steering BurdenAgents-0.51.6+1.0
ARC-AGI-1 Semi-PrivateGeneral QA94.595.5+1.0
Artificial Analysis Progress Index v1Indexes320317−1.0
LLM Chess EloGames10871107+0.8
Arena Code WebDev — Reference-Based DesignCode16141608−0.8
LMArena Agent Arena Chat Confirmed SuccessAgents4.93.6−0.7
ProgramBenchCode0.50.0−0.5
LMArena Agent Arena Chat Praise vs ComplaintAgents3.13.9+0.4
Artificial Analysis Intelligence Index v4.3.2Indexes39.539.1−0.4
Arena Alignment IndexSafety74.874.4−0.4
Bug Hunt BenchCode20.621.0+0.3
AutoformBenchMath52.852.5−0.3
VariantBench V1 Clean RerunsScience31.131.4+0.3
BrowseComp (Mercor 130-question subset)Search79.479.2−0.2
Artificial Analysis Intelligence Index v4.3Indexes39.539.4−0.1
LMArena Agent Arena Code Tool HallucinationAgents0.50.50.0
LMArena Agent Arena Work Tool HallucinationAgents0.50.50.0
LMArena Agent Arena Chat Tool HallucinationAgents0.30.30.0
Concept Synth INDUCTION Challenge100 Round 1General QA11.011.00.0
Concept Synth INDUCTION Challenge64 Round 1General QA17.217.20.0
CritPt (avg@5)Science14.314.30.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.00.00.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal20.020.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine14.714.70.0

DeepSeek V4.1 FlashvsGemini 3.7 Flashnormalised score marginsData Nov 15, 2026Data: Vals AI, Arena, Mercor and 37 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…