Atlas

Compare Models

Claude Haiku 5.5 vs GPT-6.1 Sol

GPT-6.1 Sol leads by 5.0 capability points (≈95% interval +2.6 to +7.3): a clear lead over Claude Haiku 5.5 · P(higher) ≈ 100%

Wins 144 of 182 shared benchmarks (35 lost, 3 tied)

Capability vs. relative cost
Claude Haiku 5.5GPT-6.1 SolClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Haiku 5.5vsGPT-6.1 Solcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins35 – 144 · 182 shared, 3 tied

Comparing matching setups only (48 benchmarks excluded) · suites counted once
← Claude Haiku 5.5GPT-6.1 Sol →

Claude Haiku 5.5vsGPT-6.1 Solwins by categoryData Nov 15, 2026Data: Artificial Analysis, Vals AI, Mercor and 40 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6.1 Sol minus Claude Haiku 5.5.

ARC-AGI-3 (Semi-Private)General QA2.896.2+93.4
VoxelBench Text RatingGames16962550+60.8
FrontierMath Tier 4 (v2) - PrivateMath46.3100.0+53.7
SimpleQA Verified (Epoch anti-abstention)General QA23.873.9+50.1
Mystery Game PuzzlesGames30.080.0+50.0
IOI (Vals v2)Code47.396.9+49.6
AA-PersonalityBench Informativeness indexChat & Writing75.327.7−48.1
SRE Bench — Fully SolvedCode5.750.8+45.0
Toloka Arena (v3)Agents21.066.0+45.0
NoCoT-Bench NCRI 15.2General QA68.3149+44.3
Terminal-Bench Science 0.1 (Vals)Science10.052.9+42.9
ChessBench EloGames8751562+40.9
AA-PersonalityBench Organization indexChat & Writing41.081.0+40.4
BrowseComp Extended (Mercor held-out)Search27.567.7+40.2
BrowseComp (Mercor 130-question subset)Search56.395.4+39.1
Terminal-Bench-Science 0.1Science20.058.1+38.1
RuneBenchGames4.56.9+37.2
CyberBench v1.1 OverallCybersecurity75.539.3−36.2
TaxCalcBench TY25Finance & Accounting2.038.0+36.0
ALE-BenchCode19962990+35.2
BuseyBench SVGMultimodal4.68.0+33.9
ARC-AGI-2General QA61.394.2+32.9
Furniture Assembly BenchmarkMultimodal47.580.0+32.5
AutomationBench-AA 1.0.6Agents36.066.6+30.6
NoCoT-Bench NCKI (kspine v3)General QA85.9130+30.3
Extended NYT Connections 940 (Quadratic v1)Games65.795.5+29.8
Medical Long Context Reasoning (MLCR-AA)Health & Medicine63.333.9−29.4
MMMU-Pro Extended (Mercor held-out)Multimodal31.961.3+29.4
Humanity's Last Exam DiamondGeneral QA29.658.8+29.2
ChessBenchGames41.368.7+27.3
AA-PersonalityBench Expressiveness indexChat & Writing77.250.5−26.9
Agent Initiative BenchmarkAgents6.08.7+26.0
Code Migration v1.1Code52.477.3+25.0
Decision-Bench v1.1Professional Work25.349.1+23.8
GBENCH One-shot Coding GScoreGames55.679.0+23.4
Terminal-Bench 4.0 (Artificial Analysis)Code32.856.1+23.2
LMArena Agent Arena Net Improvement (Steering Burden)Agents0.512.5+22.6
Vals Science IndexIndexes38.559.6+21.1
Code MigrationCode44.665.1+20.6
Hermes IndexIndexes37.057.1+20.1
LLM Chess EloGames11431612+19.2
Terminal-Bench 4.0Code36.455.1+18.7
FrontierMath Tiers 1-3 (v2) - PrivateMath75.193.7+18.6
Artificial Analysis Strategy & Ops IndexIndexes39.157.6+18.5
Artificial Analysis Intelligence Index v5.0Indexes36.254.5+18.3
Artificial Analysis Intelligence Index v5.0.1Indexes36.554.6+18.1
MedXpertQA MM (200-question set)Health & Medicine67.885.7+17.9
AA-3D v1.0 EloCode9981121+17.8
GameBench 2Games75.192.0+16.9
Artificial Analysis Engineering IndexIndexes42.559.4+16.8
MedXpertQA MM Extended (Mercor held-out)Health & Medicine48.464.7+16.3
CritPt v2 (agentic, avg@5)Science66.382.6+16.3
SUPERChemScience62.077.8+15.8
LMArena Agent Arena Praise vs ComplaintAgents-3.527.5+15.5
AA-Omniscience IndexIndexes10.741.5+15.4
AA-PersonalityBench Transparency indexChat & Writing77.292.4+15.4
DeepSWE v1.1Code59.975.2+15.3
Artificial Analysis Legal IndexIndexes38.753.7+15.0
ChessBench AccuracyGames66.980.7+14.8
Artificial Analysis Healthcare & Medical IndexIndexes37.552.0+14.5
Code2Video-AA EloCode11921369+14.4
MultiChallengeChat & Writing67.782.0+14.3
Roboflow Vision Evals — Reasoning, Strict (September)Multimodal72.887.0+14.2
KernelBench Mega Normalized (RTX PRO 6000)Code0.014.1+14.1
Roboflow Vision Evals — Reasoning, Judged (September)Multimodal74.888.7+13.9
LisanBench Difficulty-WeightedGames27114004+13.8
Wiz Cyber Model Arena v3.9Cybersecurity57.070.7+13.7
Artificial Analysis Economics IndexIndexes42.756.3+13.6
AA-PersonalityBench Conviction indexChat & Writing67.654.3−13.5
Roboflow Vision Evals — Detection mAP@50 (September)Multimodal68.281.6+13.4
Chess Puzzles (task version 2.0)Games55.068.0+13.0
LisanBench Path LengthGames128978736−12.9
Artificial Analysis Finance & Accounting IndexIndexes43.656.2+12.7
Design Arena Overall Code CategoriesCode12801356+12.6
Roboflow Vision Evals (October 2026, five tasks)Multimodal69.681.7+12.1
Excel Modeling BenchmarkFinance & Accounting59.070.8+11.8
LMCAGeneral QA51.162.8+11.7
MysteryMechanismScience35.146.4+11.3
GDP.pdf (Artificial Analysis, All-pass)Professional Work20.832.0+11.2
DAXBench v2Code81.992.6+10.6
EBR-bench v4 (card ban, single agent)Games43.854.3+10.5
LMArena Text Arena EloChat & Writing14221482+10.4
SpeciEval (Pooled English Runs)Safety86.997.4+10.4
Conceptual Reasoning IndexIndexes65.372.7+10.2
Roboflow Vision Evals — Detection mAP@75 (September)Multimodal54.364.3+10.0
ProofBench v1.1Math89.099.0+10.0
Humanity's Last Exam (Text-Only)General QA42.752.6+10.0
VIALS-AA (pass^3)Multimodal49.159.0+9.9
LiveBench 2.0Agents75.685.4+9.8
ARC-AGI-1 Semi-PrivateGeneral QA89.098.5+9.5
BioMysteryBench (Vals Terminus 2)Science70.479.6+9.3
CritPt (avg@5)Science22.631.7+9.1
SWE-Bench Pro V2 — FullCode87.296.1+8.9
Artificial Analysis Intelligence Index v4.3.2Indexes43.451.8+8.4
APEX-SWECode55.146.9−8.2
Terminal-Bench 2.1 Extended (Mercor held-out)Code37.745.8+8.1
BullshitBench v2 LeaderboardSafety57.065.0+8.0
SWE-bench Verified Extended (Mercor held-out)Code53.345.3−8.0
Artificial Analysis Progress Index v1Indexes326348+7.9
Roboflow Vision Evals — Detection mAP@50:95 (September)Multimodal53.060.4+7.4
Roboflow Vision Evals — Counting, Judged (September)Multimodal73.080.2+7.2
Roboflow Vision Evals — Counting, Strict (September)Multimodal73.080.2+7.2
Epoch Capabilities Index (ECI)Indexes160169+7.0
ChessBench CoherenceGames80.787.5+6.8
MKA (Artificial Analysis)Professional Work41.047.7+6.8
Vals Index v2.1Indexes54.461.2+6.7
SWE Atlas v1.1 - OverallCode59.465.9+6.5
Immigration BenchLegal61.867.8+6.0
MMMU ProMultimodal79.185.1+6.0
FrontierCode 2.0 (Artificial Analysis)Code34.940.6+5.7
AA-LCR v1.1Search78.384.0+5.7
Excel Modeling Benchmark v1.1Finance & Accounting65.571.1+5.6
LMArena Agent Arena Steering BurdenAgents2.413.4+5.5
SM BenchSafety74.369.0−5.4
Legal Research BenchLegal43.838.5−5.3
Vals Index v2.2Indexes58.763.9+5.2
LMArena Agent Arena Confirmed SuccessAgents1.611.7+5.1
DualEntry Accounting AI BenchmarkFinance & Accounting76.281.2+5.0
HardGeoBench <=10miMultimodal30.035.0+5.0
APEX-Agents 1.1Professional Work68.373.2+4.9
VIALS-AA (average of 3)Multimodal60.564.8+4.3
HealthBenchHealth & Medicine61.256.9−4.3
Harvey's Legal Agent BenchmarkLegal1.35.4+4.2
Ramp Accounting BenchFinance & Accounting42.346.4+4.1
Harvey LAB (Mercor run, undisclosed ~213-task cohort)Legal4.98.8+3.9
Public Benefits Bench v1.1Professional Work63.259.3−3.9
AA-PersonalityBench Warmth indexChat & Writing69.565.7−3.8
AA-PersonalityBench Slop indexChat & Writing10.514.3+3.8
AA-AnalystAgent (pass^5)Professional Work53.850.0−3.7
MMLU ProGeneral QA87.491.0+3.6
HealthBench ProfessionalHealth & Medicine64.868.3+3.6
Finance Agent v2.1Finance & Accounting65.461.8−3.6
GDPxProfessional Work4.98.4+3.5
Roboflow Vision Evals — Identification, Judged (September)Multimodal86.589.6+3.1
Roboflow Vision Evals — Identification, Strict (September)Multimodal80.283.3+3.1
MedScribeHealth & Medicine89.586.5−3.1
Hermes BenchAgents71.073.9+3.0
MedCodeHealth & Medicine51.548.8−2.7
DistressBenchSafety71.774.2+2.5
HardGeoBench Avg DistanceMultimodal13191239+2.5
BioSimBench 1.0Science13.015.3+2.3
CharXiv Extended (Mercor held-out)Multimodal82.985.1+2.2
Finance Agent v2Finance & Accounting54.152.0−2.1
APEX-Accounting 1.1Finance & Accounting62.264.3+2.1
HLAB-Vals-1.1Legal4.26.3+2.1
AA-PersonalityBench Imagination indexChat & Writing63.861.9−1.9
CharXiv (descriptive and reasoning combined)Multimodal94.196.0+1.9
CVDP (517-problem set)Code57.055.1−1.9
KernelBench CUDA Normalized (RTX PRO 6000)Code41.443.2+1.8
Vals Finance IndexIndexes57.455.8−1.6
AdvancedIFChat & Writing80.882.4+1.6
SWE-bench VerifiedCode88.987.3−1.6
Vals Legal IndexIndexes70.368.7−1.6
o11y-bench 1.1 Pass^3Agents87.585.9−1.6
DTBenchGeneral QA95.096.5+1.5
LMArena Agent Arena Tool HallucinationAgents-2.11.0+1.5
Vibe Code Bench v1.1Code90.488.9−1.5
HardGeoBench Country AccuracyMultimodal51.352.5+1.3
HardGeoBench <=100miMultimodal45.043.8−1.2
LMArena Agent Arena Bash ReliabilityAgents7.45.0−1.2
OTIS Mock AIME 2024-2025Math98.9100.0+1.1
AA-Briefcase v1.1 EloAgents15731555−1.0
GDPval-AA v2.1Professional Work16291607−0.9
ForecastBench Preliminary Dataset Brier IndexForecasting60.861.7+0.9
FrontierCode 1.1 Main (Artificial Analysis)Code45.546.3+0.8
SciCode 1.0.1 (Artificial Analysis)Code55.055.8+0.8
SAGEMath45.846.5+0.8
ACCoRDGeneral QA81.580.9−0.7
SAGE v1.1Math89.890.5+0.7
VIALS-AA (pass@3)Multimodal71.470.8−0.6
Prophet Arena Arena Score (Fixed Context)Forecasting85.484.8−0.5
ProgramBenchCode2.53.0+0.5
SWE-bench Multimodal (Vals v1, 480 Tasks)Code44.043.5−0.4
Terminal-Bench 2.1Code85.085.4+0.4
SWE Atlas - Codebase QnACode48.748.4−0.3
SWE-bench MultilingualCode85.184.9−0.2
Prophet Arena Arena Score (Agentic, OpenCode)Forecasting85.986.0+0.1
Tax Agent BenchFinance & Accounting62.262.3+0.1
HealthBench (length-adjusted, Artificial Analysis)Health & Medicine58.558.50.0
AIME 2025Math100.0100.00.0
ClaimBenchFinance & Accounting100.0100.00.0
τ³-Banking (Mercor run, bm25_grep retrieval)Agents43.043.00.0

Claude Haiku 5.5vsGPT-6.1 Solnormalised score marginsData Nov 15, 2026Data: Artificial Analysis, Vals AI, Mercor and 40 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…