Atlas

Compare Models

Claude Haiku 5.5 vs GPT-6 Sol

GPT-6 Sol leads by 2.4 capability points (≈95% interval +0.2 to +4.7): a clear lead over Claude Haiku 5.5 · P(higher) ≈ 99%

Wins 111 of 159 shared benchmarks (45 lost, 3 tied)

Capability vs. relative cost
Claude Haiku 5.5GPT-6 SolClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Haiku 5.5vsGPT-6 Solcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins45 – 111 · 159 shared, 3 tied

Comparing matching setups only (39 benchmarks excluded) · suites counted once
← Claude Haiku 5.5GPT-6 Sol →

Claude Haiku 5.5vsGPT-6 Solwins by categoryData Nov 15, 2026Data: Artificial Analysis, Mercor, Vals AI and 38 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6 Sol minus Claude Haiku 5.5.

Vending-Bench 2Games360214428+65.5
AA-PersonalityBench Informativeness indexChat & Writing75.310.5−65.4
Cresta-BenchAgents20.177.9+57.9
VoxelBench Text RatingGames16962394+49.7
Medical Long Context Reasoning (MLCR-AA)Health & Medicine63.316.1−47.2
AA-PersonalityBench Expressiveness indexChat & Writing77.233.4−44.2
FrontierMath Tier 4 (v2) - PrivateMath46.390.0+43.7
AA-PersonalityBench Transparency indexChat & Writing77.239.1−38.5
Toloka Arena (v3)Agents21.058.0+37.0
SimpleQA Verified (Epoch anti-abstention)General QA23.860.7+36.9
BrowseComp Extended (Mercor held-out)Search27.563.2+35.7
IOI (Vals v2)Code47.382.6+35.3
AA-PersonalityBench Imagination indexChat & Writing63.829.6−34.6
BrowseComp (Mercor 130-question subset)Search56.389.4+33.1
RuneBenchGames4.56.4+30.6
BuseyBench SVGMultimodal4.67.5+29.7
AA-PersonalityBench Organization indexChat & Writing41.012.4−28.8
ARC-AGI-2General QA61.389.6+28.3
Mystery Game PuzzlesGames30.056.0+26.0
TaxCalcBench TY25Finance & Accounting2.028.0+26.0
AutomationBench-AA 1.0.6Agents36.061.7+25.7
ChessBench EloGames8751303+25.5
NoCoT-Bench NCRI 15.2General QA68.3115+25.4
SRE Bench — Fully SolvedCode5.730.5+24.8
Extended NYT Connections 940 (Quadratic v1)Games65.790.1+24.4
NoCoT-Bench NCKI (kspine v3)General QA85.9119+22.7
ALE-BenchCode19962637+22.7
Eval Connections One-Shot with TrapsGames67.089.0+22.0
AutomationBench 1.0.6 (Zapier)Agents12.033.2+21.2
ChessBenchGames41.362.0+20.7
ARC-AGI-3 (Semi-Private)General QA2.823.0+20.2
Terminal-Bench Science 0.1 (Vals)Science10.030.0+20.0
MMMU-Pro Extended (Mercor held-out)Multimodal31.951.9+20.0
MedXpertQA MM Extended (Mercor held-out)Health & Medicine48.468.0+19.6
AA-3D v1.0 EloCode9981131+19.3
AA-PersonalityBench Conviction indexChat & Writing67.650.5−17.3
CritPt v2 (agentic, avg@5)Science64.080.9+16.9
LLM Chess EloGames11431545+16.5
Humanity's Last Exam DiamondGeneral QA29.645.2+15.6
AA-PersonalityBench Warmth indexChat & Writing69.554.3−15.4
Legal Research BenchLegal43.828.8−14.9
FrontierMath Tiers 1-3 (v2) - PrivateMath75.189.8+14.7
LisanBench Path LengthGames128978222−14.5
GBENCH One-shot Coding GScoreGames55.670.1+14.5
SWE-bench Verified Extended (Mercor held-out)Code53.339.3−14.0
MedXpertQA MM (200-question set)Health & Medicine67.881.3+13.5
HealthBenchHealth & Medicine61.248.2−13.0
Code MigrationCode44.657.2+12.6
Excel Modeling BenchmarkFinance & Accounting59.071.5+12.5
Wiz Cyber Model Arena v3.9Cybersecurity57.069.3+12.3
LMArena Agent Arena Net Improvement (Steering Burden)Agents0.56.9+12.2
SpeciEval (Pooled English Runs)Safety86.998.5+11.5
ChessBench AccuracyGames66.977.4+11.3
Decision-Bench v1.1Professional Work25.336.5+11.2
Terminal-Bench 4.0 (Artificial Analysis)Code32.843.9+11.1
Furniture Assembly BenchmarkMultimodal47.558.3+10.8
SUPERChemScience62.072.8+10.8
APEX-SWECode55.145.0−10.1
Terminal-Bench-Science 0.1Science20.030.0+10.0
AA-PersonalityBench Slop indexChat & Writing10.51.0−9.6
EBR-bench v4 (card ban, single agent)Games43.853.3+9.5
Roboflow Vision Evals — Identification, Strict (September)Multimodal80.289.6+9.4
Tax Agent BenchFinance & Accounting62.253.0−9.2
SWE-bench VerifiedCode88.980.3−8.6
DeepResearch Bench II (Mercor 130-task set)Search34.342.9+8.6
SM BenchSafety74.366.0−8.3
CritPt (avg@5)Science22.630.9+8.3
AA-Omniscience IndexIndexes10.727.1+8.2
Roboflow Vision Evals — Detection mAP@50 (September)Multimodal68.276.4+8.2
Terminal-Bench 4.0Code36.444.4+8.1
LMCAGeneral QA51.159.1+8.0
Conceptual Reasoning IndexIndexes65.371.0+7.9
Artificial Analysis Intelligence Index v5.0.1Indexes36.544.2+7.7
MedScribeHealth & Medicine89.582.0−7.5
ChessBench CoherenceGames80.788.2+7.5
HardGeoBench <=100miMultimodal45.037.5−7.5
Roboflow Vision Evals (October 2026, five tasks)Multimodal69.676.9+7.3
Roboflow Vision Evals — Identification, Judged (September)Multimodal86.593.8+7.3
GDPval Extended (Mercor held-out rubric score)Professional Work66.058.7−7.3
Hermes IndexIndexes37.044.1+7.1
LMArena Agent Arena Praise vs ComplaintAgents-3.510.0+6.7
DeepSWE v1.1Code59.966.6+6.7
Public Benefits Bench v1.1Professional Work63.256.6−6.6
ARC-AGI-1 Semi-PrivateGeneral QA89.095.5+6.5
Vals Science IndexIndexes38.545.0+6.5
AA-LCR Extended (Mercor held-out)Search63.870.2+6.4
LiveBench 2.0Agents75.681.9+6.3
LMArena Text Arena EloChat & Writing14221457+6.0
ProofBench v1.1Math89.083.0−6.0
HardGeoBench Avg DistanceMultimodal13191497−5.5
AA-LCR v1.1Search78.383.7+5.3
AA-Briefcase v1.1 EloAgents15731476−5.3
Finance Agent v2Finance & Accounting54.149.0−5.1
GDPval-AA v2.1Professional Work16291514−5.1
HardGeoBench Country AccuracyMultimodal51.346.3−5.0
MysteryMechanismScience35.130.2−5.0
Terminal-Bench 2.1 Extended (Mercor held-out)Code37.742.4+4.7
SWE Atlas - Codebase QnACode48.744.1−4.6
Roboflow Vision Evals — Detection mAP@75 (September)Multimodal54.358.8+4.5
Roboflow Vision Evals — Counting, Judged (September)Multimodal73.077.5+4.5
Roboflow Vision Evals — Counting, Strict (September)Multimodal73.077.5+4.5
Roboflow Vision Evals — Reasoning, Strict (September)Multimodal72.877.3+4.5
MedCodeHealth & Medicine51.547.1−4.5
BioMysteryBench (Vals Terminus 2)Science70.474.8+4.4
GDP.pdf (Artificial Analysis, All-pass)Professional Work20.825.2+4.4
GameBench 2Games75.179.4+4.4
LisanBench Difficulty-WeightedGames27113118+4.4
HealthBench (length-adjusted, Artificial Analysis)Health & Medicine58.554.2−4.3
CVDP (517-problem set)Code57.052.7−4.3
Artificial Analysis Intelligence Index v4.3.2Indexes43.447.6+4.2
FORTRESS Average Risk Score (ARS)Safety9.513.5−4.0
Blueprint-Bench 2Multimodal0.30.4+3.9
LMArena Agent Arena Steering BurdenAgents2.410.1+3.9
HardGeoBench <=10miMultimodal30.026.3−3.7
DistressBenchSafety71.768.0−3.7
Humanity's Last Exam (Text-Only)General QA42.746.3+3.6
HealthBench ProfessionalHealth & Medicine64.861.2−3.5
MMMU ProMultimodal79.182.5+3.4
SWE-bench MultilingualCode85.181.7−3.4
Vals Index v2.1Indexes54.457.5+3.1
Harvey LAB-AA v1.1 (hallucination-gated)Legal0.63.6+3.1
Epoch Capabilities Index (ECI)Indexes160164+3.0
BullshitBench v2 LeaderboardSafety57.054.0−3.0
Roboflow Vision Evals — Reasoning, Judged (September)Multimodal74.877.7+2.9
Artificial Analysis Progress Index v1Indexes326334+2.8
KernelBench Mega Normalized (RTX PRO 6000)Code0.02.7+2.7
SciCode 1.0.1 (Artificial Analysis)Code55.057.6+2.7
Vibe Code Bench v1.1Code90.487.8−2.6
Roboflow Vision Evals — Detection mAP@50:95 (September)Multimodal53.055.6+2.6
Hermes BenchAgents71.073.5+2.5
APEX-Accounting 1.1Finance & Accounting62.259.7−2.5
CyberBench v1.1 OverallCybersecurity75.578.0+2.5
DTBenchGeneral QA95.097.3+2.3
Design Arena Overall Code CategoriesCode12801293+2.2
GDPval (Mercor 213-task rubric score)Professional Work85.583.5−2.0
ForecastBench Preliminary Dataset Brier IndexForecasting60.862.6+1.8
LMArena Agent Arena Bash ReliabilityAgents7.410.9+1.8
LMArena Agent Arena Tool HallucinationAgents-2.11.0+1.5
MMLU ProGeneral QA87.488.9+1.5
LMArena Agent Arena Confirmed SuccessAgents1.64.5+1.5
OTIS Mock AIME 2024-2025Math98.9100.0+1.1
APEX-1 (Mercor, mean score)Professional Work64.763.6−1.1
Chess Puzzles (task version 2.0)Games55.056.0+1.0
SAGEMath45.844.8−1.0
CharXiv (descriptive and reasoning combined)Multimodal94.195.0+0.9
CharXiv Extended (Mercor held-out)Multimodal82.983.8+0.9
ACCoRDGeneral QA81.582.4+0.9
Harvey LAB (Mercor run, undisclosed ~213-task cohort)Legal4.94.1−0.8
FrontierCode 2.0 (Artificial Analysis)Code34.935.6+0.6
KernelBench CUDA Normalized (RTX PRO 6000)Code41.442.0+0.6
BioSimBench 1.0Science13.013.6+0.6
ProgramBenchCode2.52.0−0.5
Harvey's Legal Agent BenchmarkLegal1.31.7+0.4
APEX-Agents 1.1Professional Work68.368.7+0.4
Terminal-Bench 2.1Code85.085.4+0.4
AdvancedIFChat & Writing80.880.9+0.1
AIME 2025Math100.0100.00.0
DualEntry Accounting AI BenchmarkFinance & Accounting76.276.20.0
τ³-Banking (Mercor run, bm25_grep retrieval)Agents43.043.00.0

Claude Haiku 5.5vsGPT-6 Solnormalised score marginsData Nov 15, 2026Data: Artificial Analysis, Mercor, Vals AI and 38 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…