Atlas

Compare Models

Claude Haiku 5.5 vs GPT-5.6 Sol

GPT-5.6 Sol leads by 1.5 capability points (≈95% interval −0.6 to +3.7): statistically tied · P(higher) ≈ 92%

Wins 97 of 134 shared benchmarks (36 lost, 1 tied)

Capability vs. relative cost
Claude Haiku 5.5GPT-5.6 SolClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Haiku 5.5vsGPT-5.6 Solcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins36 – 97 · 134 shared, 1 tied

Comparing matching setups only (43 benchmarks excluded) · suites counted once
← Claude Haiku 5.5GPT-5.6 Sol →

Claude Haiku 5.5vsGPT-5.6 Solwins by categoryData Nov 15, 2026Data: Vals AI, Artificial Analysis, Roboflow and 40 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-5.6 Sol minus Claude Haiku 5.5.

IOI (Vals v2)Code47.391.2+43.9
VoxelBench Text RatingGames16962297+42.8
Toloka Arena (v3)Agents21.061.0+40.0
Medical Long Context Reasoning (MLCR-AA)Health & Medicine63.326.1−37.2
FrontierMath Tier 4 (v2) - PrivateMath46.382.9+36.6
Vending-Bench 2Games36029619+36.4
A-Fantasia (Valid Attempts with Retries)Games72.336.0+36.3
BrowseComp (Mercor 130-question subset)Search56.390.6+34.3
ARC-AGI-2General QA61.392.5+31.2
Extended NYT Connections 940 (Quadratic v1)Games65.793.8+28.1
TaxCalcBench TY25Finance & Accounting2.030.0+28.0
Mystery Game PuzzlesGames30.058.0+28.0
LisanBench Path LengthGames128974699−25.5
SRE Bench — Fully SolvedCode5.730.5+24.8
AutomationBench-AA 1.0.6Agents36.060.1+24.1
NoCoT-Bench NCKI (kspine v3)General QA85.9121+24.0
NoCoT-Bench NCRI 15.2General QA68.3109+22.6
RuneBenchGames4.55.9+22.1
ChessBench EloGames8751218+20.4
DeepResearch Bench II (Mercor 130-task set)Search34.351.5+17.2
AutomationBench 1.0.6 (Zapier)Agents12.028.8+16.8
LLM Chess EloGames11431550+16.7
Eval Connections One-Shot with TrapsGames67.083.0+16.0
BuseyBench SVGMultimodal4.66.1+15.5
ALE-BenchCode19962410+14.7
FrontierMath Tiers 1-3 (v2) - PrivateMath75.189.1+14.0
LisanBench Difficulty-WeightedGames27111403−14.0
Excel Modeling BenchmarkFinance & Accounting59.072.3+13.3
SM BenchSafety74.361.1−13.2
CritPt v2 (agentic, avg@5)Science64.077.1+13.1
DeepSWE v1.1Code59.972.7+12.8
SpeciEval (Pooled English Runs)Safety86.999.0+12.1
GBENCH One-shot Coding GScoreGames55.667.7+12.0
FrontierSWE V2Code43.832.2−11.6
LMArena Text Arena EloChat & Writing14221485+10.9
FORTRESS Average Risk Score (ARS)Safety9.520.1−10.7
ChessBench AccuracyGames66.976.4+10.2
Terminal-Bench Science 0.1 (Vals)Science10.020.0+10.0
Vibe Code Bench v1.1Code90.480.5−9.9
RedlineBenchLegal40.650.3+9.7
Furniture Assembly BenchmarkMultimodal47.556.7+9.2
SUPERChemScience62.071.0+9.0
BullshitBench v2 LeaderboardSafety57.048.0−9.0
ARC-AGI-1 Semi-PrivateGeneral QA89.097.5+8.5
ChessBench CoherenceGames80.789.1+8.4
Code MigrationCode44.652.9+8.3
SWE-Bench Pro V2 — FullCode87.295.5+8.3
Roboflow Vision Evals — Detection mAP@75 (September)Multimodal54.346.1−8.2
LMArena Agent Arena Net Improvement (Steering Burden)Agents0.54.6+7.9
Roboflow Vision Evals — Detection mAP@50:95 (September)Multimodal53.045.2−7.8
Finance Agent v2 - Financial ModelingFinance & Accounting28.320.7−7.6
MedCodeHealth & Medicine51.544.0−7.6
KernelBench Mega Normalized (RTX PRO 6000)Code0.07.4+7.4
LMCAGeneral QA51.158.5+7.4
ARC-AGI-3 (Semi-Private)General QA0.57.8+7.3
Wiz Cyber Model Arena v3.9Cybersecurity57.064.2+7.2
APEX-SWECode55.148.0−7.1
Terminal-Bench 4.0 (Artificial Analysis)Code32.839.9+7.1
GDP.pdf (Artificial Analysis, All-pass)Professional Work20.827.8+7.0
SAGEMath45.852.6+6.8
CursorBench 4.0Code48.441.7−6.7
Roboflow Vision Evals — Identification, Strict (September)Multimodal80.286.5+6.3
AA-AnalystAgent (pass^5)Professional Work53.847.5−6.2
Roboflow Vision Evals — Identification, Judged (September)Multimodal86.592.7+6.2
ProofBench v1.1Math89.083.0−6.0
CritPt (avg@5)Science22.628.6+6.0
HealthBenchHealth & Medicine61.255.3−5.9
Tax Agent BenchFinance & Accounting62.268.0+5.8
AA-LCR v1.1Search78.384.0+5.7
AA-Omniscience IndexIndexes10.722.0+5.6
ACCoRDGeneral QA81.575.9−5.6
AA-Briefcase v1.1 EloAgents15731479−5.2
GameBench 2Games75.180.1+5.0
LMArena Agent Arena Praise vs ComplaintAgents-3.56.3+4.9
SWE Atlas - Codebase QnACode48.753.5+4.8
BrowseComp Extended (Mercor held-out)Search27.532.3+4.8
Conceptual Reasoning IndexIndexes65.368.7+4.8
Humanity's Last Exam (Text-Only)General QA42.747.3+4.6
Terminal-Bench 2.1Code85.089.5+4.5
Legal Research BenchLegal43.848.1+4.3
MedScribeHealth & Medicine89.585.2−4.3
Chess Puzzles (task version 2.0)Games55.059.0+4.0
HardGeoBench <=100miMultimodal45.041.3−3.7
HardGeoBench Country AccuracyMultimodal51.347.5−3.7
Artificial Analysis Intelligence Index v4.3.2Indexes43.447.0+3.6
Political Science LLM BenchmarkProfessional Work65.769.2+3.6
Vals Index v2.1Indexes54.458.0+3.6
MMMU ProMultimodal79.182.7+3.6
HardGeoBench Avg DistanceMultimodal13191429−3.4
Public Benefits Bench v1.1Professional Work63.266.5+3.3
Roboflow Vision Evals — Counting, Judged (September)Multimodal73.076.1+3.1
Roboflow Vision Evals — Counting, Strict (September)Multimodal73.076.1+3.1
Roboflow Vision Evals — Reasoning, Judged (September)Multimodal74.871.7−3.1
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting61.964.9+3.1
Vals Science IndexIndexes38.541.5+3.0
Roboflow Vision Evals (October 2026, five tasks)Multimodal69.672.4+2.8
SciCode 1.0.1 (Artificial Analysis)Code55.057.8+2.8
Terminal-Bench-Science 0.1Science20.022.4+2.4
Epoch Capabilities Index (ECI)Indexes160163+2.2
FrontierCode 1.1 Extended - ScoreCode58.460.6+2.2
LMArena Agent Arena Bash ReliabilityAgents7.411.4+2.0
DistressBenchSafety71.773.6+1.9
Artificial Analysis Progress Index v1Indexes326331+1.9
MysteryMechanismScience35.133.3−1.8
CVDP (517-problem set)Code57.055.4−1.6
LMArena Agent Arena Steering BurdenAgents2.45.5+1.6
LMArena Agent Arena Tool HallucinationAgents-2.11.0+1.5
Terminal-Bench 4.0Code36.437.9+1.5
Finance Agent v2 - All PassFinance & Accounting42.941.4−1.5
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting74.475.6+1.3
Harvey's Legal Agent BenchmarkLegal1.32.5+1.3
OTIS Mock AIME 2024-2025Math98.9100.0+1.1
Roboflow Vision Evals — Reasoning, Strict (September)Multimodal72.871.7−1.1
Finance Agent v2 - Market AnalysisFinance & Accounting63.164.2+1.1
ProgramBenchCode2.51.5−1.0
EBR-bench v4 (card ban, single agent)Games43.844.8+1.0
Finance Agent v2 - Disclosure AnalysisFinance & Accounting64.363.3−0.9
Finance Agent v2 - Earnings AnalysisFinance & Accounting73.574.4+0.9
CyberBench v1.1 OverallCybersecurity75.576.3+0.8
Humanity's Last Exam DiamondGeneral QA29.630.4+0.8
Finance Agent v2 - PrecedentsFinance & Accounting27.026.2−0.8
BioMysteryBench (Vals Terminus 2)Science70.471.1+0.7
DTBenchGeneral QA95.095.7+0.7
LiveBench 2.0Agents75.676.3+0.6
Blueprint-Bench 2Multimodal0.30.3+0.6
ForecastBench Preliminary Dataset Brier IndexForecasting60.861.4+0.6
Prophet Arena Arena Score (Fixed Context)Forecasting85.484.8−0.5
GDPval-AA v2.1Professional Work16291619−0.5
LMArena Agent Arena Confirmed SuccessAgents1.62.3+0.3
Finance Agent v2 - AdjustmentsFinance & Accounting49.649.2−0.3
Terminal-Bench 2.1 Extended (Mercor held-out)Code37.737.4−0.3
Roboflow Vision Evals — Detection mAP@50 (September)Multimodal68.268.4+0.2
Finance Agent v2 - ComparablesFinance & Accounting45.345.30.0
HardGeoBench <=10miMultimodal30.030.00.0

Claude Haiku 5.5vsGPT-5.6 Solnormalised score marginsData Nov 15, 2026Data: Vals AI, Artificial Analysis, Roboflow and 40 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…