Atlas

Compare Models

Gemini 3.7 Flash vs Gemini 3.8 Flash

Gemini 3.8 Flash leads by 1.3 capability points (≈95% interval −0.8 to +3.4): statistically tied · P(higher) ≈ 89%

Wins 146 of 267 shared benchmarks (89 lost, 32 tied)

Capability vs. relative cost
Gemini 3.7 FlashGemini 3.8 FlashClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Gemini 3.7 FlashvsGemini 3.8 Flashcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins89 – 146 · 267 shared, 32 tied

Comparing matching setups only (7 benchmarks excluded) · suites counted once
← Gemini 3.7 FlashGemini 3.8 Flash →

Gemini 3.7 FlashvsGemini 3.8 Flashwins by categoryData Nov 15, 2026Data: Vals AI, Arena, Mercor and 54 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Gemini 3.8 Flash minus Gemini 3.7 Flash.

Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.040.0+40.0
SWE-bench Verified Extended (Mercor held-out)Code30.054.0+24.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal40.020.0−20.0
Harvey's Legal Agent Benchmark - InsuranceLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.020.0+20.0
LLM Chess EloGames11071546+18.0
LMArena Agent Arena Net ImprovementAgents-1.33.8+16.7
ScBench V1 Clean RerunsScience50.434.0−16.4
BrowseComp Extended (Mercor held-out)Search25.541.5+16.0
ChessBench EloGames12651527+15.6
FrontierMath Tier 4 (v2) - PrivateMath36.622.0−14.6
BrokenArXiv 04/2026Math19.333.6+14.3
SAGEMath49.235.1−14.2
AA-AnalystAgent (pass^5)Professional Work60.046.3−13.7
ClockBench AccuracyMultimodal53.366.7+13.4
BioMysteryBench - Human DifficultScience43.556.5+13.0
CadQueryEval (Corrected Geometry)Code84.096.0+12.0
RiemannBenchMath39.251.2+12.0
Olam Social Poker EloGames14681479+12.0
ChessBench CoherenceGames83.294.6+11.4
IOI (Vals v2)Code67.856.9−10.9
ArXivMath 05/2026Math47.558.3+10.8
ALE-BenchCode18762182+10.8
BrokenArXiv 05/2026Math8.519.0+10.5
𝜏³-BankingAgents35.545.8+10.3
Mystery Game PuzzlesGames37.047.0+10.0
ProofBench v1.1Math58.048.0−10.0
BrokenArXiv 06/2026Math11.120.8+9.7
LLM Creative Story WritingChat & Writing-0.70.1+9.7
Tax Agent BenchFinance & Accounting57.766.8+9.1
Agent Red Teaming k=100 (CAIS)Safety38.529.8+8.7
LMArena Agent Arena Net Improvement (Steering Burden)Agents-2.22.3+8.5
Finance Agent v2 - Financial ModelingFinance & Accounting24.232.6+8.5
OSWorld 2.0 - pre-August-8, 500 steps, maximum of three partial scoresAgents50.659.0+8.4
LLM Debate BenchmarkChat & Writing14551514+8.4
Vibe Code Bench v1.1Code70.478.7+8.3
Chess Puzzles (task version 2.0)Games66.058.0−8.0
TaxCalcBench TY25Finance & Accounting14.06.0−8.0
Eval Connections One-Shot with TrapsGames80.088.0+8.0
SkillsBench with SkillsAgents65.958.0−7.9
BioSecBench-Refusal V1 Clean RerunsSafety54.847.4−7.4
Terminal-Bench 4.0Code6.113.1+7.1
AA-LCR Extended (Mercor held-out)Search48.955.6+6.7
ArXivMath 04/2026Math57.564.2+6.7
Medical Long Context Reasoning (MLCR-AA)Health & Medicine15.021.7+6.7
BabaIsBench Stage 1: The LakeGames93.386.7−6.7
APEX-Agents 1.1Professional Work79.472.9−6.5
Bio Capabilities Index (SecureBio)Indexes162153−6.3
MBCT (SecureBio)Science55.048.9−6.1
Terminal-Bench 4.0 (Artificial Analysis)Code13.619.7+6.1
GBENCH One-shot Coding GScoreGames52.658.6+6.1
ComplexConstraintsChat & Writing42.448.4+6.0
IntPhys 2Multimodal68.262.5−5.7
APEX-SWECode42.036.4−5.6
DeepSWE v1.1Code65.571.0+5.5
CRED v0.92Science74.079.5+5.5
MedCodeHealth & Medicine53.448.1−5.3
HLE Extended (Mercor held-out)General QA31.636.8+5.2
MetagenomicsBench V1Science31.726.7−5.0
Furniture Assembly BenchmarkMultimodal26.731.7+5.0
MetagenomicsBench V0Science31.726.7−5.0
AA-Briefcase EloAgents11161202+5.0
MASKSafety53.758.6+4.9
Document ArenaProfessional Work14761482+4.9
ArXivMath 06/2026Math62.567.4+4.9
P2NPCBench (NP-complete, pass@1)Code57.862.6+4.8
ARC-AGI-2General QA84.689.2+4.6
Legal Research BenchLegal34.638.9+4.3
Antidote: Everyday EditionSafety10751087+4.3
Continual Learning Exam (Trial 1)Agents25.321.2−4.2
RedlineBenchLegal45.341.3−4.0
Shroombench (mushroom identification, top-1)Multimodal61.065.0+4.0
CritPt (avg@5)Science14.318.3+4.0
Finance Agent v2 - Market AnalysisFinance & Accounting74.578.5+4.0
BrowseComp (Mercor 130-question subset)Search79.283.1+3.9
LMArena Agent Arena Praise vs ComplaintAgents-2.94.9+3.9
Arena Code WebDev — FullstackCode16291615−3.8
SkillsBench without SkillsAgents44.941.0−3.8
MindCube TinyMultimodal88.492.2+3.8
HardGeoBench <=100miMultimodal56.360.0+3.8
Gray Swan IPI Attack Success (k=15)Safety9.25.5+3.7
Vals Index v2.1Indexes51.354.9+3.6
Arena Code WebDev — Data & AnalyticsCode15771553−3.6
Ramp Accounting BenchFinance & Accounting42.345.7+3.4
VCT v2 (SecureBio)Science48.645.2−3.4
Arena Code WebDev — Consumer ProductCode15741548−3.4
Hemingway-benchChat & Writing11051093−3.3
EnigmaEvalGeneral QA33.937.2+3.3
VoxelBench Text RatingGames18441798−3.3
Arena Code WebDev — Content Creation ToolsCode15741554−3.3
SciCode 1.0.1 (Artificial Analysis)Code59.856.6−3.2
SWE-bench MultilingualCode82.685.8+3.2
RSI-Exam 0.1 (Full 88)Agents0.30.3+3.2
FrontierMath Tiers 1-3 (v2) - PrivateMath71.668.4−3.2
Concept Synth INDUCTION Challenge64 Round 1General QA17.214.1−3.1
LMArena Agent Arena Steering BurdenAgents-7.3-1.3+3.0
Machiavelli Deception (CAIS)Safety85.586.5−3.0
GeneBench-Pro Public Release SubsetScience14.011.0−3.0
ARC-AGI-1 Semi-PrivateGeneral QA95.598.5+3.0
Vals Index v2Indexes59.362.3+2.9
VIALSMultimodal26.529.4+2.9
FrontierCode 1.1 Extended - ScoreCode56.353.5−2.8
LAB-Bench ProtocolQA (SecureBio)Science86.884.0−2.8
SpatialViz-BenchMultimodal75.378.1+2.8
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting65.368.1+2.8
FrontierCode 1.1 Extended - Pass RateCode62.359.6−2.7
LAB-Bench LitQA2 (SecureBio)Science54.251.5−2.7
BioSecBench-Surveillance V1 Clean RerunsScience41.544.1+2.6
SpatialBench V0 VerifiedScience64.367.0+2.6
EnterpriseBench: CoreCraftProfessional Work55.958.5+2.6
BioSecBench-Surveillance V0 (102 tasks)Science41.544.1+2.6
GDPval Extended (Mercor held-out rubric score)Professional Work57.860.4+2.6
LMCAGeneral QA50.452.9+2.5
TxBench Oligo Discovery V1 Clean RerunsScience35.633.1−2.5
MineBench — Bradley-Terry RatingGames17651806+2.5
VCT (SecureBio)Science49.346.9−2.4
LMArena Agent Arena Confirmed SuccessAgents-3.51.3+2.4
LMArena Agent Arena Bash RecoveryAgents-5.4-0.7+2.4
Finance Agent v2 - Earnings AnalysisFinance & Accounting79.181.4+2.3
Finance Agent v2 - PrecedentsFinance & Accounting33.836.1+2.3
BioSecBench-Function V1 Clean RerunsScience33.736.0+2.3
HPCT (SecureBio)Science64.762.4−2.3
Finance Agent v2 - All PassFinance & Accounting47.549.7+2.2
ArXivLean 06/2026Math21.719.6−2.2
Arena Code WebDev — HTMLCode15931609+2.2
LMArena Agent Arena SteerabilityAgents-0.04.3+2.1
Humanity's Last Exam DiamondGeneral QA31.229.1−2.1
ChessBench AccuracyGames79.281.1+2.0
Concept Synth INDUCTION Challenge100 Round 1General QA11.09.0−2.0
Veracode GenAI Code Security (Fall 2026)Cybersecurity60.062.0+2.0
Bug Hunt BenchCode15.217.1+1.9
Creative Writing v3 Rubric ScoreChat & Writing15.215.6+1.9
MedXpertQA MM Extended (Mercor held-out)Health & Medicine62.760.8−1.9
ERQAMultimodal77.779.6+1.9
FACTS Benchmark SuiteGeneral QA71.269.4−1.9
Terminal-Bench 2.1Code85.887.6+1.9
Artificial Analysis Intelligence Index v4.3.2Indexes39.140.9+1.9
LMArena Agent Arena Bash ReliabilityAgents6.29.9+1.8
Humanity's Sixth SenseMultimodal39.841.6+1.8
APEX-1 (Mercor, mean score)Professional Work67.068.8+1.8
DistressBenchSafety69.070.7+1.8
Artificial Analysis Intelligence Index v4.3Indexes39.441.2+1.8
Code MigrationCode34.836.5+1.7
Finance Agent v2 - ComparablesFinance & Accounting50.352.0+1.7
BioMysteryBench - Human SolvableScience87.188.8+1.7
CharXiv ReasoningMultimodal84.586.2+1.7
GDPval (Mercor 213-task rubric score)Professional Work85.283.5−1.7
LVBenchMultimodal85.487.1+1.7
Arena Code WebDev — Reference-Based DesignCode16081596−1.7
GDPval-AA v2.1Professional Work14041442+1.7
OTIS Mock AIME 2024-2025Math97.298.9+1.7
Vals RSI Index v1.1 (four tasks)Agents18.319.9+1.7
VCT-Refusal (CAIS)Safety4.86.4+1.6
APEX-Accounting 1.1Finance & Accounting57.258.8+1.6
HardGeoBench Avg DistanceMultimodal895843+1.6
Harvey HLAB — Criteria Pass RateLegal88.690.2+1.6
Conceptual Reasoning IndexIndexes64.665.7+1.5
AA-Omniscience IndexIndexes26.529.6+1.5
GDPval-AA v2Professional Work14331464+1.5
PG-LLM-50Science0.30.4+1.5
Arena Code WebDev — Brand & MarketingCode15531563+1.5
Arena Image-to-WebDevCode16001594−1.4
Terminal-Bench Science 0.1 (Vals)Science7.18.6+1.4
LMArena Text Arena EloChat & Writing14891497+1.4
Arena Alignment IndexSafety74.475.8+1.4
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine14.713.3−1.3
MortgageTax (Vals all 1,258 documents)Finance & Accounting66.765.3−1.3
Creative Writing v3 EloChat & Writing17201744+1.3
Arena Code WebDev — SimulationsCode16181607−1.2
Arena Code WebDev — ReactCode15841575−1.2
ForecastBench Preliminary Dataset Brier IndexForecasting61.760.5−1.2
MedXpertQA MM (200-question set)Health & Medicine83.584.7+1.2
TextQuests Game ProgressGames52.954.1+1.2
PCT Political Inconsistency (CAIS)Safety45.146.3−1.2
SpatialBench-Long V1 Clean RerunsScience29.828.6−1.2
ACCoRDGeneral QA77.078.2+1.2
ScBench-Long V1 Clean RerunsScience18.819.9+1.1
AutomationBench-AA 1.0.6Agents62.060.9−1.1
DTBenchGeneral QA96.895.7−1.1
Artificial Analysis Progress Index v1Indexes317320+1.0
Toloka Arena (v3)Agents58.059.0+1.0
AA-LCR v1.1Search83.084.0+1.0
GDP.pdf - all-pass rate reported by GoogleProfessional Work34.035.0+1.0
ProgramBenchCode0.01.0+1.0
WordleBenchGames97.098.0+1.0
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting81.880.9−1.0
WCB (SecureBio)Science58.057.1−0.9
Excel Modeling BenchmarkFinance & Accounting71.372.2+0.9
SpeciEval (Pooled English Runs)Safety89.089.9+0.8
LiveCodeBench (Vals AI public dataset)Code88.789.5+0.8
Remote Labor Index (RLI)Professional Work5.05.8+0.8
SpeciEval (Refusal-Aware Retries, Historical)Safety89.089.9+0.8
Design Arena Overall Code CategoriesCode13041299−0.8
DAYJOB: HealthcareHealth & Medicine0.00.8+0.8
SWE-bench VerifiedCode80.880.0−0.8
Finance Agent v2 - Disclosure AnalysisFinance & Accounting67.768.5+0.7
Android Bench 2.0Code7.38.0+0.7
TxBench Preclinical Pharmacology V1 Clean RerunsScience55.354.7−0.7
FrontierSWE V2Code20.319.6−0.6
EpiBench V1 Clean RerunsScience22.021.4−0.6
GDP.pdfProfessional Work23.823.2−0.6
Humanity's Last ExamGeneral QA45.646.2+0.6
LAB-Bench CloningScenarios (SecureBio)Science99.4100.0+0.6
MedScribeHealth & Medicine83.984.5+0.6
Arena Code WebDev — FrontendCode15921588−0.5
GPQA DiamondGeneral QA95.594.9−0.5
BioTIER RefusalSafety84.483.9−0.5
ChartographyMultimodal40.440.9+0.5
SimpleQA Verified (Epoch anti-abstention)General QA69.269.7+0.5
HieroglyphBenchMultimodal55.054.6−0.4
Arena Code WebDev — GamingCode16381635−0.4
MMMU ProMultimodal84.984.5−0.3
ATLAS-Finance v1.1Finance & Accounting3.33.7+0.3
NoCoT-Bench NCKI (kspine v3)General QA121121−0.3
Roboflow Vision Evals (October 2026, five tasks)Multimodal80.179.8−0.3
SpatialBench V1 Clean RerunsScience59.159.4+0.3
TaxEval v2Finance & Accounting74.774.4−0.3
LegalBenchLegal87.387.0−0.3
TxBench Antibody Discovery V1 Clean RerunsScience43.543.3−0.2
SUPERChemScience72.872.6−0.2
CharXiv Extended (Mercor held-out)Multimodal79.178.9−0.2
SM BenchSafety80.680.4−0.2
Vision Arena OverallMultimodal12941293−0.2
Epoch Capabilities Index (ECI)Indexes158159+0.2
FrontierCS Algorithmic v1.1Code48.047.8−0.2
NoCoT-Bench NCRI 15.2General QA126126−0.2
LiveBench 2.0Agents67.267.3+0.2
LMArena Agent Arena Tool HallucinationAgents0.70.4−0.1
Finance Agent v2 - AdjustmentsFinance & Accounting54.754.8+0.1
CharXiv (descriptive and reasoning combined)Multimodal95.395.4+0.1
MMLU ProGeneral QA90.690.7+0.1
Roboflow Vision Evals (September 2026 protocol)Multimodal85.285.1−0.1
BuseyBench SVGMultimodal6.66.6−0.1
RuneBenchGames6.06.00.0
AIME 2025Math96.796.70.0
BabaIsBench Stage 0: The IntroGames100.0100.00.0
CyberBench v1.1 OverallCybersecurity43.843.80.0
DAYJOB: FinanceFinance & Accounting2.82.80.0
HANDBOOK.mdProfessional Work10.810.80.0
HardGeoBench <=10miMultimodal46.346.30.0
HardGeoBench Country AccuracyMultimodal67.567.50.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal20.020.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal20.020.00.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal20.020.00.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal0.00.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.00.00.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal20.020.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
HPCT Refusal (CAIS)Safety13.013.00.0
MMMU-Pro Extended (Mercor held-out)Multimodal36.436.40.0
ObviousBenchGeneral QA100.0100.00.0
Terminal-Bench 2.1 Extended (Mercor held-out)Code44.444.40.0
TextQuests Average Cumulative HarmSafety24.424.40.0
VariantBench V1 Clean RerunsScience31.431.40.0

Gemini 3.7 FlashvsGemini 3.8 Flashnormalised score marginsData Nov 15, 2026Data: Vals AI, Arena, Mercor and 54 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…