Atlas

Compare Models

Claude Haiku 5.5 vs Gemini 3.8 Flash

Claude Haiku 5.5 leads by 0.6 capability points (≈95% interval −1.5 to +2.8): statistically tied · P(higher) ≈ 72%

Gemini 3.8 Flash wins 109 of 185 shared benchmarks (74 lost, 2 tied)

Capability vs. relative cost
Claude Haiku 5.5Gemini 3.8 FlashClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Haiku 5.5vsGemini 3.8 Flashcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins74 – 109 · 185 shared, 2 tied

Comparing matching setups only (34 benchmarks excluded) · suites counted once
← Claude Haiku 5.5Gemini 3.8 Flash →

Claude Haiku 5.5vsGemini 3.8 Flashwins by categoryData Nov 15, 2026Data: Vals AI, Artificial Analysis, Mercor and 41 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Gemini 3.8 Flash minus Claude Haiku 5.5.

AA-PersonalityBench Slop indexChat & Writing10.571.4+61.5
AA-PersonalityBench Transparency indexChat & Writing77.223.8−53.8
SimpleQA Verified (Epoch anti-abstention)General QA23.869.7+45.9
Medical Long Context Reasoning (MLCR-AA)Health & Medicine63.321.7−41.7
ProofBench v1.1Math89.048.0−41.0
ChessBench EloGames8751527+38.8
Toloka Arena (v3)Agents21.059.0+38.0
AA-PersonalityBench Organization indexChat & Writing41.073.3+32.7
ARC-AGI-3 (Semi-Private)General QA2.835.0+32.2
ChessBenchGames41.373.3+32.0
CyberBench v1.1 OverallCybersecurity75.543.8−31.7
Extended NYT Connections 940 (Quadratic v1)Games65.797.4+31.7
NoCoT-Bench NCRI 15.2General QA68.3126+31.5
FrontierML ($200 wallet)Agents34.95.7−29.2
ARC-AGI-2General QA61.389.2+27.9
BrowseComp (Mercor 130-question subset)Search56.383.1+26.8
AutomationBench-AA 1.0.6Agents36.060.9+24.9
FrontierMath Tier 4 (v2) - PrivateMath46.322.0−24.4
FrontierSWE V2Code43.819.6−24.1
NoCoT-Bench NCKI (kspine v3)General QA85.9121+24.1
Code2Video-AA EloCode1192899−23.8
AA-PersonalityBench Imagination indexChat & Writing63.841.0−23.1
Eval Connections One-Shot with TrapsGames67.088.0+21.0
BuseyBench SVGMultimodal4.66.6+20.7
APEX-SWECode55.136.4−18.7
AutomationBench 1.0.6 (Zapier)Agents12.029.7+17.7
AA-PersonalityBench Conviction indexChat & Writing67.684.8+17.3
Mystery Game PuzzlesGames30.047.0+17.0
MedXpertQA MM (200-question set)Health & Medicine67.884.7+16.9
Terminal-Bench 4.0Code36.419.7−16.7
LLM Chess EloGames11431546+16.6
Furniture Assembly BenchmarkMultimodal47.531.7−15.8
Roboflow Vision Evals — Identification, Strict (September)Multimodal80.295.8+15.6
Finance Agent v2 - Market AnalysisFinance & Accounting63.178.5+15.4
ChessBench AccuracyGames66.981.1+15.2
Humanity's Last Exam (Text Only)General QA30.445.0+14.6
BrowseComp Extended (Mercor held-out)Search27.541.5+14.0
ChessBench CoherenceGames80.794.6+13.9
Code Migration v1.1Code52.438.6−13.7
MineBench — Bradley-Terry RatingGames15821806+13.6
Excel Modeling BenchmarkFinance & Accounting59.072.2+13.2
LMArena Text Arena EloChat & Writing14221497+13.1
AA-Briefcase v1.1 EloAgents14411204−13.0
MedXpertQA MM Extended (Mercor held-out)Health & Medicine48.460.8+12.4
Vibe Code Bench v1.1Code90.478.7−11.8
Roboflow Vision Evals — Reasoning, Strict (September)Multimodal72.884.5+11.7
Roboflow Vision Evals — Identification, Judged (September)Multimodal86.597.9+11.4
DeepSWE v1.1Code59.971.0+11.1
GameBench 2Games75.164.0−11.1
SAGEMath45.835.1−10.7
SUPERChemScience62.072.6+10.6
Roboflow Vision Evals (October 2026, five tasks)Multimodal69.679.8+10.2
HealthBench (length-adjusted, Artificial Analysis)Health & Medicine58.548.4−10.1
Roboflow Vision Evals — Reasoning, Judged (September)Multimodal74.884.5+9.7
IOI (Vals v2)Code47.356.9+9.7
AA-PersonalityBench Warmth indexChat & Writing69.560.0−9.6
ARC-AGI-1 Semi-PrivateGeneral QA89.098.5+9.5
AA-Omniscience IndexIndexes10.729.6+9.4
Finance Agent v2 - PrecedentsFinance & Accounting27.036.1+9.2
Vending-Bench 2Games36025094+9.0
Hermes BenchAgents71.062.2−8.8
Harvey's Legal Agent BenchmarkLegal1.310.0+8.8
VIALS-AA (average of 3)Multimodal60.552.0−8.5
SWE-bench VerifiedCode88.980.6−8.3
LiveBench 2.0Agents75.667.3−8.3
Roboflow Vision Evals — Detection mAP@75 (September)Multimodal54.362.5+8.2
AA-LCR Extended (Mercor held-out)Search63.855.6−8.2
BioMysteryBench (Vals Terminus 2)Science70.462.2−8.1
SWE Atlas - Codebase QnACode48.740.6−8.1
VIALS-AA (pass^3)Multimodal49.141.0−8.1
Code MigrationCode44.636.5−8.0
HLAB-Vals-1.1Legal4.212.1+7.9
Finance Agent v2 - Earnings AnalysisFinance & Accounting73.581.4+7.9
SWE-Bench Pro V2 — FullCode87.294.9+7.6
AA-AnalystAgent (pass^5)Professional Work53.846.3−7.5
VIALS-AA (pass@3)Multimodal71.464.0−7.5
Excel Modeling Benchmark v1.1Finance & Accounting65.572.9+7.4
DualEntry Accounting AI BenchmarkFinance & Accounting76.283.2+7.0
Finance Agent v2 - All PassFinance & Accounting42.949.7+6.8
Terminal-Bench 2.1 Extended (Mercor held-out)Code37.744.4+6.7
Finance Agent v2 - ComparablesFinance & Accounting45.352.0+6.7
FrontierMath Tiers 1-3 (v2) - PrivateMath75.168.4−6.7
Roboflow Vision Evals — Detection mAP@50 (September)Multimodal68.274.8+6.6
ALE-BenchCode19962182+6.6
Harvey LAB (Mercor run, undisclosed ~213-task cohort)Legal4.911.4+6.5
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting74.480.9+6.5
Roboflow Vision Evals — Counting, Judged (September)Multimodal73.079.3+6.3
Roboflow Vision Evals — Counting, Strict (September)Multimodal73.079.3+6.3
CVDP (517-problem set)Code57.050.7−6.3
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting61.968.1+6.2
SM BenchSafety74.380.4+6.1
Roboflow Vision Evals — Detection mAP@50:95 (September)Multimodal53.059.0+6.0
τ³-Banking (Mercor run, bm25_grep retrieval)Agents43.048.8+5.8
AA-PersonalityBench Informativeness indexChat & Writing75.381.0+5.8
HealthBench ProfessionalHealth & Medicine64.859.0−5.8
MKA (Artificial Analysis)Professional Work41.035.3−5.7
Artificial Analysis Intelligence Index v5.0Indexes36.230.5−5.7
AA-LCR v1.1Search78.384.0+5.7
Artificial Analysis Intelligence Index v5.0.1Indexes36.530.8−5.6
Blueprint-Bench 2Multimodal0.30.4+5.6
GDPval Extended (Mercor held-out rubric score)Professional Work66.060.4−5.6
MMMU ProMultimodal79.184.5+5.4
DAXBench v2Code81.987.2+5.3
Finance Agent v2 - AdjustmentsFinance & Accounting49.654.8+5.3
FrontierCode 1.1 Extended - Pass RateCode64.759.6−5.1
MedScribeHealth & Medicine89.584.5−5.0
FrontierCode 1.1 Extended - ScoreCode58.453.5−5.0
Legal Research BenchLegal43.838.9−4.8
APEX-Agents 1.1Professional Work68.372.9+4.6
Tax Agent BenchFinance & Accounting62.266.8+4.6
MMMU-Pro Extended (Mercor held-out)Multimodal31.936.4+4.5
FrontierCode 2.0 (Artificial Analysis)Code23.218.7−4.5
WeirdML v3Science11.77.4−4.3
Finance Agent v2 - Financial ModelingFinance & Accounting28.332.6+4.3
CritPt (avg@5)Science22.618.3−4.3
Finance Agent v2 - Disclosure AnalysisFinance & Accounting64.368.5+4.2
LMArena Agent Arena Praise vs ComplaintAgents-3.54.9+4.2
FrontierPhysics v0.1Science4.20.0−4.2
APEX-1 (Mercor, mean score)Professional Work64.768.8+4.1
HealthBenchHealth & Medicine61.257.1−4.1
CharXiv Extended (Mercor held-out)Multimodal82.978.9−4.0
TaxCalcBench TY25Finance & Accounting2.06.0+4.0
AA-3D v1.0 EloCode998974−3.5
LMArena Agent Arena Net Improvement (Steering Burden)Agents0.52.3+3.5
Ramp Accounting BenchFinance & Accounting42.345.7+3.4
MedCodeHealth & Medicine51.548.1−3.4
APEX-Accounting 1.1Finance & Accounting62.258.8−3.4
ACCoRDGeneral QA81.578.2−3.4
Bug Hunt BenchCode20.517.1−3.3
AIME 2025Math100.096.7−3.3
MMLU ProGeneral QA87.490.7+3.3
Design Arena Overall Code CategoriesCode12801299+3.2
SWE-bench Multimodal (Vals v1, 480 Tasks)Code44.040.8−3.1
Political Science LLM BenchmarkProfessional Work65.768.7+3.0
Vals Finance IndexIndexes57.460.5+3.0
GBENCH One-shot Coding GScoreGames55.658.6+3.0
Chess Puzzles (task version 2.0)Games55.058.0+3.0
SpeciEval (Pooled English Runs)Safety86.989.9+2.9
Vals Index v2.2Indexes58.756.0−2.8
Vals Science IndexIndexes38.535.8−2.7
Terminal-Bench 2.1Code85.087.6+2.6
CritPt v2 (agentic, avg@5)Science62.665.1+2.6
Artificial Analysis Intelligence Index v4.3.2Indexes43.440.9−2.5
BioSimBench 1.0Science13.015.3+2.3
SWE Atlas v1.1 - OverallCode59.457.3−2.1
Public Benefits Bench v1.1Professional Work63.265.3+2.1
Artificial Analysis Progress Index v1Indexes326320−2.0
Terminal-Bench 4.0 (Artificial Analysis)Code21.719.7−2.0
GDP.pdf (Artificial Analysis, All-pass)Professional Work20.822.8+2.0
GDPval (Mercor 213-task rubric score)Professional Work85.583.5−2.0
AA-PersonalityBench Expressiveness indexChat & Writing77.279.1+1.9
LMCAGeneral QA51.152.9+1.8
LMArena Agent Arena Steering BurdenAgents2.4-1.3−1.8
SciCode 1.0.1 (Artificial Analysis)Code55.056.6+1.6
Immigration BenchLegal61.863.4+1.6
Decision-Bench v1.1Professional Work25.326.9+1.6
SAGE v1.1Math89.888.2−1.6
ProgramBenchCode2.51.0−1.5
LMArena Code Arena WebDevCode15941583−1.5
Terminal-Bench Science 0.1 (Vals)Science10.08.6−1.4
MysteryMechanismScience35.136.5+1.4
CharXiv (descriptive and reasoning combined)Multimodal94.195.4+1.3
Epoch Capabilities Index (ECI)Indexes160159−1.3
LMArena Agent Arena Tool HallucinationAgents-2.10.4+1.2
LMArena Agent Arena Bash ReliabilityAgents7.49.9+1.2
Hermes IndexIndexes37.038.1+1.1
DistressBenchSafety71.770.7−1.0
GDPval-AA v2.1Professional Work14241442+0.8
DTBenchGeneral QA95.095.7+0.7
SWE-bench Verified Extended (Mercor held-out)Code53.354.0+0.7
SWE-bench MultilingualCode85.185.8+0.7
RedlineBenchLegal40.641.3+0.7
Conceptual Reasoning IndexIndexes65.365.7+0.6
Humanity's Last Exam (Text-Only)General QA42.742.1−0.6
Vals Legal IndexIndexes70.369.8−0.5
Humanity's Last Exam DiamondGeneral QA29.629.1−0.5
Vals Index v2.1Indexes54.454.9+0.5
Artificial Analysis Coding Agent Index v1.5Indexes41.441.9+0.4
Finance Agent v2.1Finance & Accounting65.465.8+0.4
ForecastBench Preliminary Dataset Brier IndexForecasting60.860.5−0.3
LMArena Agent Arena Confirmed SuccessAgents1.61.3−0.1
Prophet Arena Arena Score (Fixed Context)Forecasting85.485.3−0.1
Prophet Arena Arena Score (Agentic, OpenCode)Forecasting85.986.0+0.1
OTIS Mock AIME 2024-2025Math98.998.90.0
Terminal-Bench-Science 0.1Science10.010.00.0

Claude Haiku 5.5vsGemini 3.8 Flashnormalised score marginsData Nov 15, 2026Data: Vals AI, Artificial Analysis, Mercor and 41 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…