Atlas

Compare Models

Claude Haiku 5.5 vs Gemini 3.7 Flash

Claude Haiku 5.5 leads by 1.9 capability points (≈95% interval −0.1 to +3.9): statistically tied · P(higher) ≈ 97%

Gemini 3.7 Flash wins 84 of 158 shared benchmarks (56 lost, 18 tied)

Capability vs. relative cost
Claude Haiku 5.5Gemini 3.7 FlashClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Haiku 5.5vsGemini 3.7 Flashcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins56 – 84 · 158 shared, 18 tied

Comparing matching setups only (26 benchmarks excluded) · suites counted once
← Claude Haiku 5.5Gemini 3.7 Flash →

Claude Haiku 5.5vsGemini 3.7 Flashwins by categoryData Nov 15, 2026Data: Vals AI, Mercor, Arena and 27 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Gemini 3.7 Flash minus Claude Haiku 5.5.

Medical Long Context Reasoning (MLCR-AA)Health & Medicine63.315.0−48.3
SimpleQA Verified (Epoch anti-abstention)General QA23.869.2+45.4
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal0.040.0+40.0
Toloka Arena (v3)Agents21.058.0+37.0
CyberBench v1.1 OverallCybersecurity75.543.8−31.7
NoCoT-Bench NCRI 15.2General QA68.3126+31.7
ProofBench v1.1Math89.058.0−31.0
Terminal-Bench 4.0Code33.36.1−27.2
AutomationBench-AA 1.0.6Agents36.062.0+26.1
NoCoT-Bench NCKI (kspine v3)General QA85.9121+24.4
FrontierSWE V2Code43.820.3−23.5
ARC-AGI-2General QA61.384.6+23.3
SWE-bench Verified Extended (Mercor held-out)Code53.330.0−23.3
ChessBench EloGames8751265+23.2
BrowseComp (Mercor 130-question subset)Search56.379.2+22.9
Furniture Assembly BenchmarkMultimodal47.526.7−20.8
BuseyBench SVGMultimodal4.66.6+20.8
IOI (Vals v2)Code47.367.8+20.6
Vibe Code Bench v1.1Code90.470.4−20.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.020.0+20.0
AA-Briefcase v1.1 EloAgents14411115−17.9
MedXpertQA MM (200-question set)Health & Medicine67.883.5+15.7
Roboflow Vision Evals — Identification, Strict (September)Multimodal80.295.8+15.6
Arena Code WebDev — FullstackCode15711629+15.3
Harvey HLAB — Criteria Pass RateLegal73.488.6+15.2
AA-LCR Extended (Mercor held-out)Search63.848.9−14.9
MedXpertQA MM Extended (Mercor held-out)Health & Medicine48.462.7+14.3
ChessBench AccuracyGames66.979.2+13.2
APEX-SWECode55.142.0−13.1
Eval Connections One-Shot with TrapsGames67.080.0+13.0
Excel Modeling BenchmarkFinance & Accounting59.071.3+12.3
TaxCalcBench TY25Finance & Accounting2.014.0+12.0
LMArena Text Arena EloChat & Writing14221489+11.7
Finance Agent v2 - Market AnalysisFinance & Accounting63.174.5+11.4
APEX-Agents 1.1Professional Work68.379.4+11.1
MineBench — Bradley-Terry RatingGames15821765+11.1
Chess Puzzles (task version 2.0)Games55.066.0+11.0
SUPERChemScience62.072.8+10.8
Roboflow Vision Evals (October 2026, five tasks)Multimodal69.680.1+10.5
Roboflow Vision Evals — Identification, Judged (September)Multimodal86.596.9+10.4
Arena Code WebDev — Data & AnalyticsCode15071577+10.4
Code MigrationCode44.634.8−9.8
FrontierMath Tier 4 (v2) - PrivateMath46.336.6−9.8
Legal Research BenchLegal43.834.6−9.1
DeepResearch Bench II (Mercor 130-task set)Search34.343.4+9.1
LiveBench 2.0Agents75.667.2−8.5
Roboflow Vision Evals — Reasoning, Strict (September)Multimodal72.881.2+8.4
CritPt (avg@5)Science22.614.3−8.3
GDPval Extended (Mercor held-out rubric score)Professional Work66.057.8−8.2
Terminal-Bench 4.0 (Artificial Analysis)Code21.713.6−8.1
AA-Omniscience IndexIndexes10.726.5+7.9
SWE-bench VerifiedCode88.981.0−7.9
Roboflow Vision Evals — Detection mAP@75 (September)Multimodal54.362.1+7.8
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting74.481.8+7.4
Arena Code WebDev — Consumer ProductCode15161574+7.4
Roboflow Vision Evals — Reasoning, Judged (September)Multimodal74.881.9+7.1
Mystery Game PuzzlesGames30.037.0+7.0
Finance Agent v2 - PrecedentsFinance & Accounting27.033.8+6.8
Terminal-Bench 2.1 Extended (Mercor held-out)Code37.744.4+6.7
ARC-AGI-1 Semi-PrivateGeneral QA89.095.5+6.5
Roboflow Vision Evals — Counting, Judged (September)Multimodal73.079.3+6.3
Roboflow Vision Evals — Counting, Strict (September)Multimodal73.079.3+6.3
SM BenchSafety74.380.6+6.3
AA-AnalystAgent (pass^5)Professional Work53.860.0+6.3
Roboflow Vision Evals — Detection mAP@50 (September)Multimodal68.274.3+6.1
Roboflow Vision Evals — Detection mAP@50:95 (September)Multimodal53.058.9+5.9
MMMU ProMultimodal79.184.9+5.8
MedScribeHealth & Medicine89.583.9−5.6
DeepSWE v1.1Code59.965.5+5.6
Finance Agent v2 - Earnings AnalysisFinance & Accounting73.579.1+5.6
Finance Agent v2 - AdjustmentsFinance & Accounting49.654.7+5.2
LMArena Agent Arena Net Improvement (Steering Burden)Agents0.5-2.2−5.0
APEX-Accounting 1.1Finance & Accounting62.257.2−5.0
Finance Agent v2 - ComparablesFinance & Accounting45.350.3+5.0
SciCode 1.0.1 (Artificial Analysis)Code55.059.8+4.9
LMArena Agent Arena Steering BurdenAgents2.4-7.3−4.9
RedlineBenchLegal40.645.3+4.7
AA-LCR v1.1Search78.383.0+4.7
Arena Code WebDev — GamingCode16791638−4.6
Finance Agent v2 - All PassFinance & Accounting42.947.5+4.6
ACCoRDGeneral QA81.577.0−4.6
Tax Agent BenchFinance & Accounting62.257.7−4.5
MMMU-Pro Extended (Mercor held-out)Multimodal31.936.4+4.5
Artificial Analysis Intelligence Index v4.3.2Indexes43.439.1−4.3
Arena Code WebDev — HTMLCode15631593+4.3
ALE-BenchCode19961876−4.2
Arena Code WebDev — ReactCode16151584−4.2
Finance Agent v2 - Financial ModelingFinance & Accounting28.324.2−4.2
Design Arena Overall Code CategoriesCode12801304+4.0
CharXiv Extended (Mercor held-out)Multimodal82.979.1−3.8
Humanity's Last Exam (Text-Only)General QA42.739.0−3.7
FrontierMath Tiers 1-3 (v2) - PrivateMath75.171.6−3.5
Finance Agent v2 - Disclosure AnalysisFinance & Accounting64.367.7+3.5
SAGEMath45.849.2+3.4
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting61.965.3+3.4
AIME 2025Math100.096.7−3.3
MMLU ProGeneral QA87.490.6+3.2
Vals Index v2.1Indexes54.451.3−3.2
Artificial Analysis Progress Index v1Indexes326317−3.1
GBENCH One-shot Coding GScoreGames55.652.6−3.0
Terminal-Bench Science 0.1 (Vals)Science10.07.1−2.9
GDP.pdf (Artificial Analysis, All-pass)Professional Work20.823.6+2.8
DistressBenchSafety71.769.0−2.7
LMArena Agent Arena Confirmed SuccessAgents1.6-3.5−2.5
ChessBench CoherenceGames80.783.2+2.5
ProgramBenchCode2.50.0−2.5
SWE-bench MultilingualCode85.182.6−2.5
FrontierCode 1.1 Extended - Pass RateCode64.762.3−2.4
APEX-1 (Mercor, mean score)Professional Work64.767.0+2.3
FrontierCode 1.1 Extended - ScoreCode58.456.3−2.2
Wiz Cyber Model Arena v3.9Cybersecurity57.059.1+2.1
SpeciEval (Pooled English Runs)Safety86.989.0+2.1
BrowseComp Extended (Mercor held-out)Search27.525.5−2.0
MedCodeHealth & Medicine51.553.4+1.8
DTBenchGeneral QA95.096.8+1.8
OTIS Mock AIME 2024-2025Math98.997.2−1.7
Arena Code WebDev — SimulationsCode16321618−1.6
Humanity's Last Exam DiamondGeneral QA29.631.2+1.6
Arena Code WebDev — FrontendCode16041592−1.5
LLM Chess EloGames11431107−1.5
Epoch Capabilities Index (ECI)Indexes160158−1.5
LMArena Agent Arena Tool HallucinationAgents-2.10.7+1.4
CharXiv (descriptive and reasoning combined)Multimodal94.195.3+1.2
Conceptual Reasoning IndexIndexes65.364.6−1.0
Arena Code WebDev — Content Creation ToolsCode15681574+1.0
ForecastBench Preliminary Dataset Brier IndexForecasting60.861.7+0.9
GDPval-AA v2.1Professional Work14241404−0.9
Terminal-Bench 2.1Code85.085.8+0.8
LMCAGeneral QA51.150.4−0.7
Arena Code WebDev — Brand & MarketingCode15571553−0.6
LMArena Agent Arena Bash ReliabilityAgents7.46.2−0.6
Bug Hunt BenchCode20.521.0+0.5
Arena Code WebDev — Reference-Based DesignCode16061608+0.3
LMArena Agent Arena Praise vs ComplaintAgents-3.5-2.9+0.3
GDPval (Mercor 213-task rubric score)Professional Work85.585.2−0.3
Ramp Accounting BenchFinance & Accounting42.342.30.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal0.00.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.00.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.00.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0

Claude Haiku 5.5vsGemini 3.7 Flashnormalised score marginsData Nov 15, 2026Data: Vals AI, Mercor, Arena and 27 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…