Atlas

Compare Models

Claude Haiku 5.5 vs GPT-6 Astra

GPT-6 Astra leads by 5.9 capability points (≈95% interval +3.5 to +8.3): a clear lead over Claude Haiku 5.5 · P(higher) ≈ 100%

Wins 160 of 215 shared benchmarks (32 lost, 23 tied)

Capability vs. relative cost
Claude Haiku 5.5GPT-6 AstraClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Haiku 5.5vsGPT-6 Astracapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins32 – 160 · 215 shared, 23 tied

Comparing matching setups only (36 benchmarks excluded) · suites counted once
← Claude Haiku 5.5GPT-6 Astra →

Claude Haiku 5.5vsGPT-6 Astrawins by categoryData Nov 15, 2026Data: Vals AI, Artificial Analysis, Mercor and 46 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6 Astra minus Claude Haiku 5.5.

ARC-AGI-3 (Semi-Private)General QA2.898.6+95.7
VoxelBench Text RatingGames16962625+66.2
Mystery Game PuzzlesGames30.084.0+54.0
Terminal-Bench Science 0.1 (Vals)Science10.062.9+52.9
IOI (Vals v2)Code47.3100.0+52.7
ChessBench EloGames8751755+52.3
SimpleQA Verified (Epoch anti-abstention)General QA23.875.6+51.8
FrontierMath Tier 4 (v2) - PrivateMath46.397.6+51.3
SRE Bench — Fully SolvedCode5.756.9+51.1
NoCoT-Bench NCRI 15.2General QA68.3159+49.8
AA-PersonalityBench Informativeness indexChat & Writing75.329.6−46.2
Toloka Arena (v3)Agents21.067.0+46.0
RuneBenchGames4.57.3+43.1
TaxCalcBench TY25Finance & Accounting2.044.0+42.0
ALE-BenchCode19963164+41.4
BrowseComp (Mercor 130-question subset)Search56.394.2+37.9
Terminal-Bench-Science 0.1Science18.154.8+36.7
CyberBench v1.1 OverallCybersecurity75.541.1−34.4
Humanity's Last Exam DiamondGeneral QA29.663.8+34.2
ARC-AGI-2General QA61.395.0+33.7
AutomationBench-AA 1.0.6Agents36.068.5+32.5
Furniture Assembly BenchmarkMultimodal47.580.0+32.5
Extended NYT Connections 940 (Quadratic v1)Games65.798.1+32.4
EBR-bench v4 (card ban, single agent)Games43.876.2+32.4
NoCoT-Bench NCKI (kspine v3)General QA85.9131+31.3
Terminal-Bench 4.0 (Artificial Analysis)Code29.359.6+30.3
AutomationBench 1.0.6 (Zapier)Agents12.041.4+29.4
Agent Initiative BenchmarkAgents6.09.0+29.2
AA-PersonalityBench Organization indexChat & Writing41.069.5+28.8
ChessBenchGames41.370.0+28.7
Medical Long Context Reasoning (MLCR-AA)Health & Medicine63.335.0−28.3
Code Migration v1.1Code52.480.6+28.2
MMMU-Pro Extended (Mercor held-out)Multimodal31.960.0+28.1
BuseyBench SVGMultimodal4.67.3+27.6
Eval Connections One-Shot with TrapsGames67.094.0+27.0
Vals Science IndexIndexes38.565.1+26.6
LMArena Agent Arena Net Improvement (Steering Burden)Agents0.514.5+26.6
Decision-Bench v1.1Professional Work25.351.5+26.2
AA-3D v1.0 EloCode9981168+24.7
ChartographyMultimodal46.471.0+24.6
Humanity's Last Exam (Text Only)General QA30.454.2+23.8
GBENCH One-shot Coding GScoreGames55.679.2+23.6
Terminal-Bench 4.0Code36.459.6+23.2
Code MigrationCode44.667.7+23.2
AA-PersonalityBench Conviction indexChat & Writing67.644.8−23.1
RedlineBenchLegal40.662.8+22.2
FrontierSWE V2 Worst@5Code34.555.1+20.6
Artificial Analysis Intelligence Index v5.0.1Indexes36.556.9+20.4
FrontierSWE V2 Best@5Code52.873.0+20.3
Harvey's Legal Agent Benchmark - Corporate M&ALegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.020.0+20.0
ChessBench AccuracyGames66.985.0+19.4
LLM Chess EloGames11431614+19.3
GameBench 2Games75.194.3+19.3
Artificial Analysis Engineering IndexIndexes42.561.8+19.3
Hermes IndexIndexes37.056.3+19.3
AA-PersonalityBench Expressiveness indexChat & Writing77.258.1−19.2
Code2Video-AA EloCode11921427+19.0
Artificial Analysis Intelligence Index v5.0Indexes36.255.3+19.0
MedXpertQA MM (200-question set)Health & Medicine67.886.8+19.0
Chess Puzzles (task version 2.0)Games55.074.0+19.0
Artificial Analysis Strategy & Ops IndexIndexes39.158.0+18.9
FrontierMath Tiers 1-3 (v2) - PrivateMath75.193.7+18.6
SUPERChemScience62.080.3+18.3
MysteryMechanismScience35.153.2+18.0
LMArena Agent Arena Praise vs ComplaintAgents-3.531.7+17.6
AA-PersonalityBench Transparency indexChat & Writing77.294.3+17.3
CritPt v2 (agentic, avg@5)Science66.383.4+17.1
LisanBench Path LengthGames128977393−17.1
Artificial Analysis Legal IndexIndexes38.755.6+16.9
Roboflow Vision Evals — Reasoning, Strict (September)Multimodal72.889.6+16.8
Blueprint-Bench 2Multimodal0.30.5+16.7
AA-Omniscience IndexIndexes10.743.7+16.5
Roboflow Vision Evals — Reasoning, Judged (September)Multimodal74.891.2+16.4
ChessBench CoherenceGames80.797.0+16.3
Artificial Analysis Healthcare & Medical IndexIndexes37.553.7+16.2
DeepResearch Bench II (Mercor 130-task set)Search34.350.1+15.8
Roboflow Vision Evals — Detection mAP@50 (September)Multimodal68.283.6+15.4
DAXBench v2Code81.996.8+14.9
Artificial Analysis Economics IndexIndexes42.757.2+14.5
DeepSWE v1.1Code59.974.1+14.2
Harvey HLAB — Criteria Pass RateLegal73.487.5+14.1
Artificial Analysis Finance & Accounting IndexIndexes43.657.4+13.8
Roboflow Vision Evals (October 2026, five tasks)Multimodal69.683.4+13.8
VIALS-AA (pass^3)Multimodal49.162.7+13.7
AA-PersonalityBench Imagination indexChat & Writing63.877.2+13.5
LMCAGeneral QA51.164.4+13.3
Wiz Cyber Model Arena v3.9Cybersecurity57.070.2+13.2
Roboflow Vision Evals — Detection mAP@75 (September)Multimodal54.367.0+12.7
HardGeoBench Avg DistanceMultimodal1319908+12.7
Excel Modeling BenchmarkFinance & Accounting59.071.7+12.7
MedXpertQA MM Extended (Mercor held-out)Health & Medicine48.460.8+12.4
BullshitBench v2 LeaderboardSafety57.069.0+12.0
Conceptual Reasoning IndexIndexes65.373.9+11.8
GDP.pdf (Artificial Analysis, All-pass)Professional Work20.832.2+11.4
LiveBench 2.0Agents75.686.1+10.5
Humanity's Last Exam (Text-Only)General QA42.753.1+10.4
SWE Atlas v1.1 - OverallCode59.469.4+10.0
ProofBench v1.1Math89.099.0+10.0
SWE-Bench Pro V2 — FullCode87.296.9+9.7
SpeciEval (Pooled English Runs)Safety86.996.5+9.6
ARC-AGI-1 Semi-PrivateGeneral QA89.098.5+9.5
MKA (Artificial Analysis)Professional Work41.050.2+9.3
Artificial Analysis Intelligence Index v4.3.2Indexes43.452.7+9.3
CritPt (avg@5)Science22.631.7+9.1
LMArena Text Arena EloChat & Writing14221474+9.0
Roboflow Vision Evals — Detection mAP@50:95 (September)Multimodal53.061.9+8.9
BioMysteryBench (Vals Terminus 2)Science70.479.3+8.9
HardGeoBench Country AccuracyMultimodal51.360.0+8.8
OSWorld 2.1 offline - reported partial reward, step limit unspecifiedAgents72.480.9+8.5
Artificial Analysis Progress Index v1Indexes326350+8.2
Roboflow Vision Evals — Counting, Judged (September)Multimodal73.081.1+8.1
Roboflow Vision Evals — Counting, Strict (September)Multimodal73.081.1+8.1
VIALS-AA (average of 3)Multimodal60.568.3+7.9
Finance Agent v2 - ComparablesFinance & Accounting45.337.8−7.6
HardGeoBench <=10miMultimodal30.037.5+7.5
FrontierCode 2.0 (Artificial Analysis)Code34.942.4+7.4
MMMU ProMultimodal79.186.4+7.3
Vibe Code Bench 1–100Code20.427.6+7.2
Excel Modeling Benchmark v1.1Finance & Accounting65.572.7+7.2
Epoch Capabilities Index (ECI)Indexes160169+7.2
APEX-Agents 1.1Professional Work68.375.1+6.8
Finance Agent v2 - Financial ModelingFinance & Accounting28.321.6−6.7
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting61.968.4+6.5
APEX-SWECode55.148.6−6.5
Ramp Accounting BenchFinance & Accounting42.348.7+6.4
Roboflow Vision Evals — Identification, Strict (September)Multimodal80.286.5+6.3
HardGeoBench <=100miMultimodal45.051.3+6.2
LMArena Agent Arena Steering BurdenAgents2.414.8+6.2
LMArena Agent Arena Confirmed SuccessAgents1.614.0+6.2
Roboflow Vision Evals — Identification, Judged (September)Multimodal86.592.7+6.2
FrontierCode 1.1 Extended - ScoreCode58.464.5+6.1
Immigration BenchLegal61.867.7+5.9
AA-PersonalityBench Warmth indexChat & Writing69.575.3+5.8
AA-PersonalityBench Slop indexChat & Writing10.516.2+5.8
DualEntry Accounting AI BenchmarkFinance & Accounting76.281.2+5.0
Finance Agent v2.1Finance & Accounting65.460.4−4.9
HealthBench ProfessionalHealth & Medicine64.869.7+4.9
Terminal-Bench 2.1Code85.089.9+4.9
GDPxProfessional Work4.99.6+4.7
APEX-1 (Mercor, mean score)Professional Work64.769.3+4.6
SM BenchSafety74.369.9−4.4
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting74.478.8+4.4
Legal Research BenchLegal43.839.4−4.3
LisanBench Difficulty-WeightedGames27113113+4.3
HealthBenchHealth & Medicine61.256.9−4.3
Harvey LAB (Mercor run, undisclosed ~213-task cohort)Legal4.99.2+4.3
MMLU ProGeneral QA87.491.5+4.1
Finance Agent v2 - Earnings AnalysisFinance & Accounting73.569.5−4.0
τ³-Banking (Mercor run, bm25_grep retrieval)Agents43.046.7+3.7
Finance Agent v2 - PrecedentsFinance & Accounting27.030.3+3.3
o11y-bench 1.1 Pass^3Agents87.584.4−3.1
MedCodeHealth & Medicine51.548.5−3.1
ProgramBenchCode2.55.5+3.0
HLAB-Vals-1.1Legal4.27.1+2.9
BioSimBench 1.0Science13.015.9+2.9
Hermes BenchAgents71.073.7+2.7
FORTRESS Average Risk Score (ARS)Safety9.512.1−2.7
Finance Agent v2 - All PassFinance & Accounting42.940.3−2.6
DistressBenchSafety71.774.3+2.6
Finance Agent v2 - Disclosure AnalysisFinance & Accounting64.361.7−2.6
AA-AnalystAgent (pass^5)Professional Work53.851.3−2.5
APEX-Accounting 1.1Finance & Accounting62.264.7+2.5
Finance Agent v2 - Market AnalysisFinance & Accounting63.165.5+2.5
DTBenchGeneral QA95.097.3+2.3
CharXiv Extended (Mercor held-out)Multimodal82.985.1+2.2
CVDP (517-problem set)Code57.054.9−2.1
ForecastBench Preliminary Dataset Brier IndexForecasting60.862.9+2.1
CharXiv (descriptive and reasoning combined)Multimodal94.196.1+2.0
Vals Legal IndexIndexes70.368.4−1.9
GDPval Extended (Mercor held-out rubric score)Professional Work66.064.1−1.9
VIALS-AA (pass@3)Multimodal71.473.3+1.9
GDPval-AA v2.1Professional Work16291587−1.9
Vals Finance IndexIndexes57.455.6−1.8
AA-LCR v1.1Search78.380.0+1.7
MedScribeHealth & Medicine89.587.9−1.6
LMArena Agent Arena Tool HallucinationAgents-2.11.0+1.5
SciCode 1.0.1 (Artificial Analysis)Code55.056.5+1.5
Finance Agent v2 - AdjustmentsFinance & Accounting49.648.3−1.2
AdvancedIFChat & Writing80.882.0+1.2
GDPval (Mercor 213-task rubric score)Professional Work85.586.7+1.2
Tax Agent BenchFinance & Accounting62.263.3+1.1
OTIS Mock AIME 2024-2025Math98.9100.0+1.1
SWE Atlas - Codebase QnACode48.747.8−0.9
Prophet Arena Arena Score (Fixed Context)Forecasting85.484.7−0.7
SWE-bench Multimodal (Vals v1, 480 Tasks)Code44.043.3−0.6
SAGEMath45.846.4+0.6
Prophet Arena Arena Score (Agentic, OpenCode)Forecasting85.986.2+0.3
HealthBench (length-adjusted, Artificial Analysis)Health & Medicine58.558.2−0.3
LMArena Agent Arena Bash ReliabilityAgents7.47.8+0.2
AA-Briefcase v1.1 EloAgents15731569−0.2
ACCoRDGeneral QA81.581.7+0.2
AIME 2025Math100.0100.00.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.00.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.00.00.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal0.00.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0

Claude Haiku 5.5vsGPT-6 Astranormalised score marginsData Nov 15, 2026Data: Vals AI, Artificial Analysis, Mercor and 46 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…