Atlas

Compare Models

Claude Haiku 5.5 vs Opus 4.8

Claude Haiku 5.5 leads by 1.6 capability points (≈95% interval −0.4 to +3.6): statistically tied · P(higher) ≈ 94%

Wins 75 of 130 shared benchmarks (54 lost, 1 tied)

Capability vs. relative cost
Claude Haiku 5.5Opus 4.8Closed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Haiku 5.5vsOpus 4.8capability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins75 – 54 · 130 shared, 1 tied

Comparing matching setups only (23 benchmarks excluded) · suites counted once
← Claude Haiku 5.5Opus 4.8 →

Claude Haiku 5.5vsOpus 4.8wins by categoryData Nov 15, 2026Data: Mercor, Vals AI, Artificial Analysis and 36 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Opus 4.8 minus Claude Haiku 5.5.

Roboflow Vision Evals — Detection mAP@75 (September)Multimodal54.313.3−41.0
KernelBench Mega Normalized (RTX PRO 6000)Code0.040.6+40.6
A-Fantasia (Valid Attempts with Retries)Games72.334.7+37.7
BullshitBench v2 LeaderboardSafety57.094.0+37.0
Roboflow Vision Evals — Detection mAP@50:95 (September)Multimodal53.016.8−36.2
Roboflow Vision Evals — Detection mAP@50 (September)Multimodal68.238.6−29.6
SimpleQA Verified (Epoch anti-abstention)General QA23.853.0+29.2
LLM Chess EloGames1143450−28.4
Roboflow Vision Evals — Reasoning, Strict (September)Multimodal72.844.4−28.4
Extended NYT Connections 940 (Quadratic v1)Games65.791.1+25.4
Roboflow Vision Evals — Reasoning, Judged (September)Multimodal74.853.0−21.8
NoCoT-Bench NCKI (kspine v3)General QA85.9117+21.7
Roboflow Vision Evals — Counting, Strict (September)Multimodal73.052.7−20.3
Roboflow Vision Evals — Counting, Judged (September)Multimodal73.054.0−19.0
Blueprint-Bench 2Multimodal0.30.1−18.5
NoCoT-Bench NCRI 15.2General QA68.3102+18.5
TaxCalcBench TY25Finance & Accounting2.020.0+18.0
Medical Long Context Reasoning (MLCR-AA)Health & Medicine63.345.6−17.8
BrowseComp (Mercor 130-question subset)Search56.373.7+17.4
KernelBench CUDA Normalized (RTX PRO 6000)Code41.424.9−16.5
FrontierCode 2.0 - Pass RateCode31.917.1−14.7
AA-Briefcase v1.1 EloAgents15731320−13.8
FrontierCode 2.0 - ScoreCode28.915.3−13.6
Terminal-Bench 4.0Code36.423.2−13.1
GameBench 2Games75.162.5−12.6
MineBench — Bradley-Terry RatingGames15821786+12.4
Roboflow Vision Evals (October 2026, five tasks)Multimodal69.658.3−11.3
APEX-SWECode55.143.9−11.2
Terminal-Bench 4.0 (Artificial Analysis)Code32.821.7−11.1
ARC-AGI-2General QA61.372.1+10.8
LisanBench Path LengthGames128979463−10.7
LMArena Text Arena EloChat & Writing14221482+10.4
Excel Modeling BenchmarkFinance & Accounting59.069.4+10.4
DeepResearch Bench II (Mercor 130-task set)Search34.344.6+10.3
SWE-bench Verified Extended (Mercor held-out)Code53.363.3+10.0
FrontierMath Tier 4 (v2) - PrivateMath46.356.1+9.8
AutomationBench-AA 1.0.6Agents36.045.6+9.6
AA-Omniscience IndexIndexes10.728.8+9.0
SAGEMath45.854.8+9.0
AA-AnalystAgent (pass^5)Professional Work53.845.0−8.7
HardGeoBench <=100miMultimodal45.036.3−8.7
Wiz Cyber Model Arena v3.9Cybersecurity57.065.4+8.4
Harvey's Legal Agent BenchmarkLegal1.39.6+8.3
Vibe Code Bench v1.1Code90.482.7−7.7
GDPval-AA v2.1Professional Work16291461−7.4
GDPval Extended (Mercor held-out rubric score)Professional Work66.058.9−7.1
AdvancedIFChat & Writing80.873.7−7.1
BuseyBench SVGMultimodal4.65.2+6.5
Humanity's Last Exam DiamondGeneral QA29.623.1−6.5
LMCAGeneral QA51.157.5+6.4
CharXiv Extended (Mercor held-out)Multimodal82.976.7−6.2
Bug Hunt BenchCode20.514.3−6.2
Mystery Game PuzzlesGames30.036.0+6.0
ChessBench AccuracyGames66.972.4+5.9
Terminal-Bench Science 0.1 (Vals)Science10.04.3−5.7
DeepSWE v1.1Code59.954.4−5.5
Roboflow Vision Evals — Identification, Strict (September)Multimodal80.275.0−5.2
ChessBench EloGames875961+5.1
LMArena Code Arena WebDevCode15941555−5.0
Furniture Assembly BenchmarkMultimodal47.542.5−5.0
Terminal-Bench 2.1 Extended (Mercor held-out)Code37.732.7−5.0
Finance Agent v2 - Financial ModelingFinance & Accounting28.323.3−5.0
Public Benefits Bench v1.1Professional Work63.268.1+4.9
VoxelBench Text RatingGames16961627−4.9
FrontierMath Tiers 1-3 (v2) - PrivateMath75.180.0+4.9
AutomationBench 1.0.6 (Zapier)Agents12.016.9+4.9
MedXpertQA MM (200-question set)Health & Medicine67.872.5+4.7
Design Arena Overall Code CategoriesCode12801252−4.6
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting61.966.4+4.5
HealthBench ProfessionalHealth & Medicine64.860.3−4.5
ExploitGymCybersecurity9.413.8+4.4
SWE Atlas - Codebase QnACode48.744.4−4.3
ACCoRDGeneral QA81.577.4−4.2
Conceptual Reasoning IndexIndexes65.368.3+4.1
Finance Agent v2 - Market AnalysisFinance & Accounting63.167.2+4.1
APEX-Agents 1.1Professional Work68.364.4−3.9
MedScribeHealth & Medicine89.585.8−3.8
ALE-BenchCode19962102+3.7
ARC-AGI-1 Semi-PrivateGeneral QA89.092.5+3.5
MedXpertQA MM Extended (Mercor held-out)Health & Medicine48.451.6+3.2
Chess Puzzles (task version 2.0)Games55.052.0−3.0
Terminal-Bench 2.1Code85.082.0−3.0
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting74.471.6−2.7
Artificial Analysis Progress Index v1Indexes326318−2.7
Tax Agent BenchFinance & Accounting62.264.9+2.7
Code MigrationCode44.647.3+2.7
SpeciEval (Pooled English Runs)Safety86.984.3−2.6
MMLU ProGeneral QA87.490.0+2.6
HardGeoBench Country AccuracyMultimodal51.353.8+2.5
BrowseComp Extended (Mercor held-out)Search27.530.0+2.5
HardGeoBench <=10miMultimodal30.027.5−2.5
HealthBenchHealth & Medicine61.258.8−2.4
HardGeoBench Avg DistanceMultimodal13191245+2.3
Humanity's Last Exam (Text-Only)General QA42.744.8+2.1
Roboflow Vision Evals — Identification, Judged (September)Multimodal86.584.4−2.1
APEX-1 (Mercor, mean score)Professional Work64.762.7−2.0
GDP.pdf (Artificial Analysis, All-pass)Professional Work20.822.8+2.0
AIME 2025Math100.098.3−1.7
SUPERChemScience62.060.3−1.7
MedCodeHealth & Medicine51.553.2+1.7
GBENCH One-shot Coding GScoreGames55.654.0−1.6
Artificial Analysis Intelligence Index v4.3.2Indexes43.441.8−1.6
ProgramBenchCode2.51.0−1.5
Finance Agent v2 - Disclosure AnalysisFinance & Accounting64.365.7+1.5
Finance Agent v2 - Earnings AnalysisFinance & Accounting73.572.1−1.4
SM BenchSafety74.372.9−1.4
Finance Agent v2 - PrecedentsFinance & Accounting27.025.6−1.3
ARC-AGI-3 (Semi-Private)General QA2.81.5−1.3
Finance Agent v2 - AdjustmentsFinance & Accounting49.648.4−1.2
FrontierCode 1.1 Extended - ScoreCode58.459.6+1.2
LiveBench 2.0Agents75.676.8+1.1
ChessBench CoherenceGames80.779.7−1.0
GDPval (Mercor 213-task rubric score)Professional Work85.584.5−1.0
RedlineBenchLegal40.639.7−0.9
CharXiv (descriptive and reasoning combined)Multimodal94.193.2−0.9
ForecastBench Preliminary Dataset Brier IndexForecasting60.861.6+0.8
FrontierCode 1.1 Extended - Pass RateCode64.765.5+0.8
Vals Index v2.1Indexes54.455.1+0.7
Finance Agent v2 - All PassFinance & Accounting42.943.5+0.6
AA-LCR Extended (Mercor held-out)Search63.863.2−0.6
SciCode 1.0.1 (Artificial Analysis)Code55.054.4−0.6
OTIS Mock AIME 2024-2025Math98.998.3−0.6
MMMU-Pro Extended (Mercor held-out)Multimodal31.931.4−0.5
Humanity's Last Exam (with tools)General QA57.457.9+0.5
Finance Agent v2 - ComparablesFinance & Accounting45.344.9−0.5
Epoch Capabilities Index (ECI)Indexes160160−0.3
DistressBenchSafety71.771.4−0.3
LisanBench Difficulty-WeightedGames27112694−0.2
DTBenchGeneral QA95.094.9−0.1
Legal Research BenchLegal43.843.80.0

Claude Haiku 5.5vsOpus 4.8normalised score marginsData Nov 15, 2026Data: Mercor, Vals AI, Artificial Analysis and 36 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…