Atlas

Compare Models

Claude Haiku 5.5 vs Grok 4.7

Claude Haiku 5.5 leads by 0.7 capability points (≈95% interval −1.4 to +2.8): statistically tied · P(higher) ≈ 75%

Wins 96 of 209 shared benchmarks (92 lost, 21 tied)

Capability vs. relative cost
Claude Haiku 5.5Grok 4.7Closed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Haiku 5.5vsGrok 4.7capability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins96 – 92 · 209 shared, 21 tied

Comparing matching setups only (20 benchmarks excluded) · suites counted once
← Claude Haiku 5.5Grok 4.7 →

Claude Haiku 5.5vsGrok 4.7wins by categoryData Nov 15, 2026Data: Vals AI, Artificial Analysis, Mercor and 45 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Grok 4.7 minus Claude Haiku 5.5.

ProofBench v1.1Math89.026.0−63.0
Medical Long Context Reasoning (MLCR-AA)Health & Medicine63.315.0−48.3
AA-PersonalityBench Informativeness indexChat & Writing75.333.4−42.3
Vending-Bench 2Games360210537+42.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.040.0+40.0
SimpleQA Verified (Epoch anti-abstention)General QA23.856.0+32.2
Roboflow Vision Evals — Detection mAP@75 (September)Multimodal54.323.7−30.6
AutomationBench-AA 1.0.6Agents36.065.6+29.6
FrontierMath Tier 4 (v2) - PrivateMath46.317.1−29.3
AA-PersonalityBench Imagination indexChat & Writing63.835.3−28.8
Roboflow Vision Evals — Detection mAP@50:95 (September)Multimodal53.024.4−28.6
Roboflow Vision Evals — Detection mAP@50 (September)Multimodal68.241.2−27.0
Furniture Assembly BenchmarkMultimodal47.520.8−26.7
DeepSWE v1.1Code59.933.3−26.6
Terminal-Bench 2.1 Extended (Mercor held-out)Code37.711.8−25.9
AA-PersonalityBench Warmth indexChat & Writing69.544.8−25.0
HardGeoBench Avg DistanceMultimodal13192104−24.2
FrontierML ($200 wallet)Agents34.910.9−24.0
FrontierMath Tiers 1-3 (v2) - PrivateMath75.153.0−22.1
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.020.0+20.0
BrowseComp (Mercor 130-question subset)Search56.375.8+19.5
DistressBenchSafety71.752.4−19.3
AA-PersonalityBench Transparency indexChat & Writing77.258.1−19.2
Eval Connections One-Shot with TrapsGames67.085.0+18.0
KernelBench Mega Normalized (RTX PRO 6000)Code0.018.0+18.0
HardGeoBench <=100miMultimodal45.027.5−17.5
Harvey HLAB — Criteria Pass RateLegal73.490.5+17.1
MineBench — Bradley-Terry RatingGames15821848+16.1
Artificial Analysis Strategy & Ops IndexIndexes39.154.8+15.7
Agent Initiative BenchmarkAgents6.04.5−15.4
AA-PersonalityBench Conviction indexChat & Writing67.682.9+15.4
Artificial Analysis Coding Agent Index v1.5Indexes41.456.3+14.8
DeepResearch Bench II (Mercor 130-task set)Search34.349.0+14.7
Harvey LAB (Mercor run, undisclosed ~213-task cohort)Legal4.919.5+14.6
FrontierSWE V2Code43.829.5−14.3
HLAB-Vals-1.1Legal4.218.3+14.2
Chess Puzzles (task version 2.0)Games55.041.0−14.0
LLM Chess EloGames1143822−13.2
HardGeoBench <=10miMultimodal30.017.5−12.5
BrowseComp Extended (Mercor held-out)Search27.539.5+12.0
SM BenchSafety74.386.3+12.0
ChessBench EloGames8751076+12.0
DualEntry Accounting AI BenchmarkFinance & Accounting76.288.1+11.9
AA-PersonalityBench Organization indexChat & Writing41.029.6−11.5
Roboflow Vision Evals — Counting, Judged (September)Multimodal73.061.7−11.3
Roboflow Vision Evals — Counting, Strict (September)Multimodal73.061.7−11.3
VIALS-AA (pass^3)Multimodal49.137.9−11.2
Extended NYT Connections 940 (Quadratic v1)Games65.776.8+11.1
AA-Omniscience IndexIndexes10.732.0+10.7
DAXBench v2Code81.992.6+10.6
IOI (Vals v2)Code47.357.7+10.4
τ³-Banking (Mercor run, bm25_grep retrieval)Agents43.032.6−10.4
Toloka Arena (v3)Agents21.031.0+10.0
MysteryMechanismScience35.125.2−9.9
SWE-bench VerifiedCode88.979.1−9.8
SpeciEval (Pooled English Runs)Safety86.996.4+9.4
o11y-bench 1.1 Pass^3Agents85.976.6−9.4
Artificial Analysis Healthcare & Medical IndexIndexes37.546.6+9.1
RedlineBenchLegal40.649.5+8.9
GDPval-AA v2.1Professional Work15231717+8.5
Artificial Analysis Legal IndexIndexes38.747.2+8.5
Artificial Analysis Finance & Accounting IndexIndexes43.651.7+8.1
VIALS-AA (average of 3)Multimodal60.552.4−8.1
Excel Modeling BenchmarkFinance & Accounting59.067.0+8.0
Roboflow Vision Evals — Reasoning, Judged (September)Multimodal74.866.9−7.9
ChessBench AccuracyGames66.974.0+7.6
HardGeoBench Country AccuracyMultimodal51.343.8−7.5
ACCoRDGeneral QA81.574.0−7.5
ARC-AGI-3 (Semi-Private)General QA2.810.0+7.2
Bug Hunt BenchCode20.527.6+7.1
HealthBenchHealth & Medicine61.254.2−7.0
APEX-Accounting 1.1Finance & Accounting62.255.3−6.9
CritPt v2 (agentic, avg@5)Science62.655.7−6.9
Tax Agent BenchFinance & Accounting62.269.0+6.8
Terminal-Bench-Science 0.1Science18.111.4−6.7
Immigration BenchLegal61.868.0+6.3
AA-Briefcase v1.1 EloAgents15231636+6.2
LMArena Code Arena WebDevCode15941641+6.2
ALE-BenchCode19961823−6.1
Roboflow Vision Evals — Reasoning, Strict (September)Multimodal72.866.7−6.1
Roboflow Vision Evals (October 2026, five tasks)Multimodal69.663.5−6.1
Excel Modeling Benchmark v1.1Finance & Accounting65.571.5+6.0
CyberBench v1.1 OverallCybersecurity75.569.5−6.0
Humanity's Last Exam DiamondGeneral QA29.623.6−6.0
TaxCalcBench TY25Finance & Accounting2.08.0+6.0
DeepFinanceFinance & Accounting40.034.1−5.9
ChessBench CoherenceGames80.786.3+5.6
VIALS-AA (pass@3)Multimodal71.465.8−5.6
Code2Video-AA EloCode11921127−5.3
SAGEMath45.840.8−5.0
Finance Agent v2.1Finance & Accounting65.460.6−4.8
CharXiv Extended (Mercor held-out)Multimodal82.978.2−4.7
CritPt (avg@5)Science22.618.0−4.6
AA-LCR Extended (Mercor held-out)Search63.868.2+4.4
SWE-Marathon v1.1Code50.045.6−4.4
VulcanBench Frontier v4 (functional pass@1)Code95.7100.0+4.3
Terminal-Bench 4.0Code33.337.6+4.3
Vibe Code Bench v1.1Code90.486.2−4.3
LMArena Agent Arena Net Improvement (Steering Burden)Agents0.52.6+3.9
KernelBench CUDA Normalized (RTX PRO 6000)Code41.445.2+3.8
Vals Science IndexIndexes38.534.8−3.7
MedXpertQA MM Extended (Mercor held-out)Health & Medicine48.452.0+3.6
Terminal-Bench 4.0 (Artificial Analysis)Code29.325.8−3.5
LMArena Text Arena EloChat & Writing14221442+3.4
Hermes BenchAgents71.074.4+3.4
Epoch Capabilities Index (ECI)Indexes160156−3.4
Legal Research BenchLegal43.847.1+3.4
LMArena Agent Arena Praise vs ComplaintAgents-3.53.1+3.3
Artificial Analysis Intelligence Index v4.3.2Indexes43.446.4+3.1
Artificial Analysis Economics IndexIndexes42.745.7+3.0
Conceptual Reasoning IndexIndexes65.363.1−3.0
SciCode 1.0.1 (Artificial Analysis)Code55.057.8+2.8
GameBench 2Games75.172.3−2.8
SWE Atlas v1.1 - OverallCode59.462.1+2.7
HealthBench (length-adjusted, Artificial Analysis)Health & Medicine58.556.0−2.5
Public Benefits Bench v1.1Professional Work63.265.6+2.4
GDP.pdf (Artificial Analysis, All-pass)Professional Work20.823.2+2.4
Vals Legal IndexIndexes70.372.7+2.4
Hermes IndexIndexes37.039.3+2.3
LMArena Agent Arena Steering BurdenAgents2.4-2.3−2.3
CursorBench 4.0Code48.446.3−2.1
CVDP (517-problem set)Code57.054.9−2.1
CharXiv (descriptive and reasoning combined)Multimodal94.192.0−2.1
SWE-bench Verified Extended (Mercor held-out)Code53.355.3+2.0
ProgramBenchCode2.50.5−2.0
MedCodeHealth & Medicine51.549.6−2.0
AA-PersonalityBench Expressiveness indexChat & Writing77.275.3−1.9
AA-PersonalityBench Slop indexChat & Writing10.58.6−1.9
SRE Bench — Fully SolvedCode5.73.8−1.9
Finance Agent v2Finance & Accounting54.152.3−1.9
Ramp Accounting BenchFinance & Accounting42.344.1+1.8
LMArena Agent Arena Confirmed SuccessAgents1.65.1+1.7
SWE Atlas - Codebase QnACode48.747.0−1.7
APEX-SWECode55.153.4−1.7
LMCAGeneral QA51.149.4−1.7
AA-LCR v1.1Search78.380.0+1.7
Design Arena Overall Code CategoriesCode12801270−1.7
LiveBench 2.0Agents75.674.1−1.5
LMArena Agent Arena Tool HallucinationAgents-2.11.0+1.5
MMLU ProGeneral QA87.488.8+1.4
Vals Finance IndexIndexes57.458.8+1.4
AA-AnalystAgent (pass^5)Professional Work53.855.0+1.3
Artificial Analysis Progress Index v1Indexes326323−1.2
MedXpertQA MM (200-question set)Health & Medicine67.869.0+1.2
ARC-AGI-1 Semi-PrivateGeneral QA89.090.2+1.2
SAGE v1.1Math89.888.7−1.1
BioMysteryBench (Vals Terminus 2)Science70.469.3−1.1
Roboflow Vision Evals — Identification, Strict (September)Multimodal80.281.3+1.1
GDPxProfessional Work4.93.9−1.1
SWE-bench Multimodal (Vals v1, 480 Tasks)Code44.042.9−1.0
DTBenchGeneral QA95.096.0+1.0
Mystery Game PuzzlesGames30.029.0−1.0
Roboflow Vision Evals — Identification, Judged (September)Multimodal86.587.5+1.0
Artificial Analysis Intelligence Index v5.0.1Indexes36.535.5−1.0
LMArena Agent Arena Bash ReliabilityAgents7.49.3+1.0
MMMU ProMultimodal79.178.2−0.9
FrontierCode 1.1 Extended - ScoreCode58.459.4+0.9
SWE-bench MultilingualCode85.184.2−0.9
Artificial Analysis Intelligence Index v5.0Indexes36.235.3−0.9
Code Migration v1.1Code52.453.2+0.9
OTIS Mock AIME 2024-2025Math98.998.1−0.8
AA-3D v1.0 EloCode998992−0.8
MKA (Artificial Analysis)Professional Work41.040.1−0.8
Artificial Analysis Engineering IndexIndexes42.541.8−0.8
APEX-1 (Mercor, mean score)Professional Work64.765.4+0.7
BioSimBench 1.0Science13.012.3−0.7
GBENCH One-shot Coding GScoreGames55.656.3+0.6
AdvancedIFChat & Writing80.880.2−0.6
FrontierCode 1.1 Extended - Pass RateCode64.765.2+0.5
Vals Index v2.1Indexes54.454.9+0.5
Blueprint-Bench 2Multimodal0.30.3−0.5
Humanity's Last Exam (Text-Only)General QA42.743.1+0.5
FrontierCode 2.0 - Pass RateCode31.932.3+0.4
APEX-Agents 1.1Professional Work68.368.7+0.4
Code MigrationCode44.644.8+0.2
RCM-Bench-1Health & Medicine88.488.2−0.2
MedScribeHealth & Medicine89.589.4−0.2
ARC-AGI-2General QA61.361.4+0.1
Vals Index v2.2Indexes58.758.6−0.1
BuseyBench SVGMultimodal4.64.6+0.1
ForecastBench Preliminary Dataset Brier IndexForecasting60.860.7−0.1
FrontierCode 2.0 - ScoreCode28.928.90.0
AIME 2025Math100.0100.00.0
FrontierPhysics v0.1Science4.24.20.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.00.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
MazeBench ASCIIGames1.01.00.0
Terminal-Bench 2.1Code85.085.00.0
Terminal-Bench Science 0.1 (Vals)Science10.010.00.0

Claude Haiku 5.5vsGrok 4.7normalised score marginsData Nov 15, 2026Data: Vals AI, Artificial Analysis, Mercor and 45 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…