Atlas

Compare Models

DeepSeek V4.1 Flash 1017 vs Gemini 3.7 Flash

DeepSeek V4.1 Flash 1017 leads by 1.6 capability points (≈95% interval −0.7 to +3.8): statistically tied · P(higher) ≈ 91%

Wins 61 of 119 shared benchmarks (45 lost, 13 tied)

Capability vs. relative cost
DeepSeek V4.1 Flash 1017Gemini 3.7 FlashClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

DeepSeek V4.1 Flash 1017vsGemini 3.7 Flashcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins61 – 45 · 119 shared, 13 tied

Comparing matching setups only (8 benchmarks excluded) · suites counted once
← DeepSeek V4.1 Flash 1017Gemini 3.7 Flash →

DeepSeek V4.1 Flash 1017vsGemini 3.7 Flashwins by categoryData Nov 15, 2026Data: Vals AI, Arena, Artificial Analysis and 23 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Gemini 3.7 Flash minus DeepSeek V4.1 Flash 1017.

SimpleQA Verified (Epoch anti-abstention)General QA42.569.2+26.7
Terminal-Bench 4.0Code31.06.1−24.9
NoCoT-Bench NCRI 15.2General QA81.6126+24.4
BuseyBench SVGMultimodal4.36.6+22.9
ZeroBench (Main questions, pass@5, with tools)Multimodal51.030.0−21.0
Chess Puzzles (task version 2.0)Games46.066.0+20.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal20.040.0+20.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - ImmigrationLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal20.00.0−20.0
Bug Hunt BenchCode40.721.0−19.7
Vibe Code Bench v1.1Code87.770.4−17.3
Tax Agent BenchFinance & Accounting74.957.7−17.2
Arena Code WebDev — Brand & MarketingCode16671553−16.6
AA-Briefcase v1.1 EloAgents14161115−16.5
Concept Synth INDUCTION Challenge64 Round 1General QA32.817.2−15.6
Terminal-Bench 4.0 (Artificial Analysis)Code28.813.6−15.2
VoxelBench Text RatingGames20471844−14.5
Medical Long Context Reasoning (MLCR-AA, October 2026 methodology)Health & Medicine26.712.2−14.4
Arena Code WebDev — HTMLCode16961593−14.4
NoCoT-Bench NCKI (kspine v3)General QA102121+13.5
Arena Code WebDev — FullstackCode16761629−12.3
Arena Code WebDev — ReactCode16751584−12.2
LiveBench 2.0Agents79.267.2−12.0
Mystery Game PuzzlesGames49.037.0−12.0
Finance Agent v2 - Market AnalysisFinance & Accounting62.774.5+11.8
Arena Code WebDev — Content Creation ToolsCode16471574−11.5
DeepSWE v1.1Code77.065.5−11.5
AA-Omniscience IndexIndexes4.726.5+10.9
Arena Code WebDev — FrontendCode16751592−10.6
Legal Research BenchLegal45.234.6−10.6
Humanity's Last Exam DiamondGeneral QA20.831.2+10.4
VCT v2 (SecureBio)Science38.548.6+10.1
Concept Synth INDUCTION Challenge100 Round 1General QA21.011.0−10.0
FrontierMath Tier 4 (v2) - PrivateMath46.336.6−9.8
CritPt (avg@5)Science24.014.3−9.7
Humanity's Sixth SenseMultimodal30.139.8+9.7
Bio Capabilities Index (SecureBio)Indexes149162+9.6
Excel Modeling BenchmarkFinance & Accounting61.871.3+9.5
Arena Code WebDev — SimulationsCode17011618−9.4
Code MigrationCode44.234.8−9.4
Agents' Last Exam - release-reported pass rate, subset unspecifiedAgents35.426.3−9.1
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting74.165.3−8.8
WCB (SecureBio)Science49.658.0+8.4
Arena Code WebDev — Reference-Based DesignCode16691608−8.1
LMArena Agent Arena Net Improvement (Steering Burden)Agents2.0-2.2−7.9
Finance Agent v2 - PrecedentsFinance & Accounting26.533.8+7.4
DAYJOB: Finance (October 2026 rescoring)Finance & Accounting12.55.3−7.2
Arena Code WebDev — GamingCode17021638−7.1
APEX-SWECode49.142.0−7.1
ARC-AGI-2General QA78.184.6+6.5
AutomationBench-AA 1.0.6Agents68.562.0−6.5
GDP.pdf (Artificial Analysis, All-pass)Professional Work17.223.6+6.4
VCT (SecureBio)Science42.949.3+6.4
GPQA-Bio (SecureBio)Science87.881.6−6.2
Artificial Analysis Intelligence Index v4.3.2Indexes45.139.1−6.0
LMArena Agent Arena Steering BurdenAgents4.0-7.3−5.7
Vals Index v2.1Indexes56.651.3−5.3
LAB-Bench LitQA2 (SecureBio)Science49.054.2+5.2
MMMU ProMultimodal79.784.9+5.1
LAB-Bench ProtocolQA (SecureBio)Science81.986.8+4.9
ALE-BenchCode20121876−4.8
Arena Code WebDev — Consumer ProductCode16111574−4.7
LAB-Bench CloningScenarios (SecureBio)Science94.899.4+4.6
HPCT (SecureBio)Science60.564.7+4.2
SciCode 1.0.1 (Artificial Analysis)Code55.959.8+3.9
Finance Agent v2 - Earnings AnalysisFinance & Accounting75.379.1+3.8
Finance Agent v2 - AdjustmentsFinance & Accounting58.454.7−3.7
DAYJOB: Healthcare (October 2026 rescoring)Health & Medicine6.02.4−3.6
Arena Code WebDev — Data & AnalyticsCode16001577−3.4
CharXiv Reasoning (with tools)Multimodal85.688.7+3.1
Humanity's Last ExamGeneral QA42.645.6+3.0
LMArena Agent Arena Confirmed SuccessAgents2.3-3.5−2.9
Finance Agent v2 - ComparablesFinance & Accounting47.550.3+2.7
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting79.281.8+2.6
EnterpriseBench: CoreCraftProfessional Work58.555.9−2.6
Design Arena Overall Code CategoriesCode12891304+2.5
Artificial Analysis Progress Index v1Indexes324317−2.2
OTIS Mock AIME 2024-2025Math99.497.2−2.2
Finance Agent v2 - All PassFinance & Accounting45.247.5+2.2
APEX-Accounting 1.1Finance & Accounting59.357.2−2.1
LMArena Agent Arena Praise vs ComplaintAgents1.2-2.9−2.0
Arena Image-to-WebDevCode16091600−1.9
LMArena Agent Arena Bash ReliabilityAgents2.46.2+1.9
Finance Agent v2 - Disclosure AnalysisFinance & Accounting66.367.7+1.5
Finance Agent v2 - Financial ModelingFinance & Accounting22.824.2+1.4
APEX-1 (Mercor, mean score)Professional Work68.367.0−1.3
Wiz Cyber Model Arena v3.9Cybersecurity60.359.1−1.2
GBENCH One-shot Coding GScoreGames53.852.6−1.2
Harvey HLAB — Criteria Pass RateLegal89.788.6−1.1
MBCT (SecureBio)Science53.955.0+1.1
ARC-AGI-1 Semi-PrivateGeneral QA94.595.5+1.0
Epoch Capabilities Index (ECI)Indexes159158−0.7
MMLU-Bio (SecureBio)Science90.791.4+0.7
RedlineBenchLegal44.945.3+0.4
FrontierMath Tiers 1-3 (v2) - PrivateMath71.271.6+0.4
AA-LCR v1.1Search82.783.0+0.3
LMArena Agent Arena Tool HallucinationAgents0.30.7+0.2
APEX-Agents 1.1Professional Work79.479.40.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal20.020.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0

DeepSeek V4.1 Flash 1017vsGemini 3.7 Flashnormalised score marginsData Nov 15, 2026Data: Vals AI, Arena, Artificial Analysis and 23 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…