Atlas

Compare Models

Claude Sonnet 5.5 vs Grok 4.7

Claude Sonnet 5.5 leads by 6.0 capability points (≈95% interval +3.6 to +8.4): a clear lead over Grok 4.7 · P(higher) ≈ 100%

Wins 214 of 269 shared benchmarks (37 lost, 18 tied)

Capability vs. relative cost
Claude Sonnet 5.5Grok 4.7Closed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Sonnet 5.5vsGrok 4.7capability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins214 – 37 · 269 shared, 18 tied

Comparing matching setups only (11 benchmarks excluded) · suites counted once
← Claude Sonnet 5.5Grok 4.7 →

Claude Sonnet 5.5vsGrok 4.7wins by categoryData Nov 15, 2026Data: Vals AI, Artificial Analysis, Mercor and 68 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Grok 4.7 minus Claude Sonnet 5.5.

ProofBench v1.1Math100.026.0−74.0
KernelBench Mega Normalized (RTX PRO 6000)Code90.618.0−72.6
ARC-AGI-3 (Semi-Private)General QA72.47.0−65.4
FrontierMath Tier 4 (v2) - PrivateMath80.517.1−63.4
Medical Long Context Reasoning (MLCR-AA)Health & Medicine75.015.0−60.0
AA-PersonalityBench Organization indexChat & Writing84.829.6−55.8
Furniture Assembly BenchmarkMultimodal75.020.8−54.2
AA-PersonalityBench Informativeness indexChat & Writing84.833.4−51.9
AA-PersonalityBench Imagination indexChat & Writing84.835.3−50.0
ChartographyMultimodal61.614.7−46.9
HardGeoBench Avg DistanceMultimodal6832104−43.8
HoneyBench v0.2 (hack rate)Safety41.685.1−43.5
AA-PersonalityBench Transparency indexChat & Writing10058.1−42.3
KernelBench CUDA Normalized (RTX PRO 6000)Code87.045.2−41.9
Terminal-Bench-Science 0.1Science52.411.4−41.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.040.0+40.0
FrontierML ($200 wallet)Agents50.410.9−39.5
LAB-Bench LitQA2 (SecureBio)Science55.816.9−38.9
DeepSWE v1.1Code70.533.3−37.2
Terminal-Bench Science 0.1 (Vals)Science47.110.0−37.1
LAB-Bench ProtocolQA (SecureBio)Science88.351.4−36.9
Mystery Game PuzzlesGames65.029.0−36.0
FrontierMath Tiers 1-3 (v2) - PrivateMath88.853.0−35.8
Arena Image-to-WebDevCode17441589−33.9
Terminal-Bench 2.1 Extended (Mercor held-out)Code45.511.8−33.7
Code Migration v1.1Code86.953.2−33.7
Toloka Arena (v3)Agents64.031.0−33.0
FrontierSWE V2Code61.929.5−32.4
OpenWorldEngineer (Mean@8)Code39.07.0−32.0
Terminal-Bench 4.0 (Artificial Analysis)Code57.125.8−31.3
AA-3D v1.0 EloCode1204992−30.7
Humanity's Last Exam DiamondGeneral QA53.323.6−29.7
SWE Atlas Codebase QnA Extended (Mercor held-out)Code44.715.3−29.4
GeneBench-Pro Public Release SubsetScience39.010.0−29.0
AA-PersonalityBench Warmth indexChat & Writing73.344.8−28.8
DAYJOB: Data ScienceProfessional Work35.36.7−28.6
MatFlowBench Pass@1Science50.022.8−27.2
DeepsecBench — Revalidated V1 (233 issues)Code37.510.9−26.6
sudo L7Code41.715.6−26.1
Code2Video-AA EloCode14431127−25.7
IOI (Vals v2)Code83.157.7−25.3
LMArena Agent Arena Net Improvement (Steering Burden)Agents16.02.6−25.3
Code MigrationCode69.844.8−25.0
AA-PersonalityBench Conviction indexChat & Writing58.182.9+25.0
Bio Capabilities Index (SecureBio)Indexes175141−24.5
Terminal-Bench 4.0Code61.837.6−24.2
MysteryMechanismScience49.125.2−23.9
TreasuryQA BenchProfessional Work41.918.1−23.8
CritPt v2 (agentic, avg@5)Science75.151.4−23.7
HPCT (SecureBio)Science77.253.7−23.5
Real-SWE 1.1Code51.228.0−23.2
VIALSMultimodal41.018.2−22.8
ARC-AGI-2General QA83.961.4−22.5
HardGeoBench <=100miMultimodal50.027.5−22.5
DistressBenchSafety74.652.4−22.2
ALE-BenchCode24481823−22.1
Short Form Bench v1.0Games23552578+22.1
ChessBench CoherenceGames64.686.3+21.7
Agent Initiative BenchmarkAgents6.74.5−21.4
τ³-Banking (Mercor run, bm25_grep retrieval)Agents54.032.6−21.4
Bug Hunt BenchCode48.927.6−21.3
SAFE-Teen 106 worst branch: Urgent Safety (no prompt)Safety23.244.4−21.2
Harvey HLAB — Criteria Pass RateLegal70.090.5+20.5
DeepFinanceFinance & Accounting54.634.1−20.5
ScBench-Long V1 Clean RerunsScience31.811.4−20.5
Concept Synth INDUCTION Challenge100 Round 1General QA44.024.0−20.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal20.00.0−20.0
BuseyBench SVGMultimodal6.54.6−19.6
Olam Social Poker EloGames15261508−19.6
AA-PersonalityBench Slop indexChat & Writing27.78.6−19.2
Conceptual Reasoning IndexIndexes77.163.1−19.2
RiemannBench (November 2026 rescoring)Math76.857.6−19.2
GameBench 2Games91.472.3−19.1
Artificial Analysis Intelligence Index v5.0.1Indexes54.535.5−19.1
Artificial Analysis Intelligence Index v5.0Indexes54.235.3−18.9
LMCAGeneral QA68.349.4−18.9
HardGeoBench <=10miMultimodal36.317.5−18.7
Chess Puzzles (task version 2.0)Games59.041.0−18.0
SWE-bench VerifiedCode96.779.1−17.6
ViBench 2.0Code19.11.5−17.6
LMArena Code Arena WebDevCode17731641−17.2
LAB-Bench CloningScenarios (SecureBio)Science97.080.1−16.9
Artificial Analysis Engineering IndexIndexes58.541.8−16.7
Concept Synth INDUCTION Challenge64 Round 1General QA50.034.4−15.6
SWE-bench Multimodal (Vals v1, 480 Tasks)Code58.342.9−15.4
LLM Chess EloGames1195822−15.3
DAYJOB: AccountingFinance & Accounting35.220.0−15.2
APEX-Agents 1.1Professional Work83.768.7−15.0
HardGeoBench Country AccuracyMultimodal58.843.8−15.0
Physical-Engineering's Last Exam (hidden test)Professional Work45.130.2−14.9
SAFE-Teen 106 worst branch: Urgent Safety (under-18 prompt)Safety11.826.7−14.9
SAFE-Teen 106: Relationships (no prompt)Safety16.931.5−14.6
SAFE-Teen 106: Health Advice (no prompt)Safety33.218.7+14.5
Roboflow Vision Evals (October 2026, five tasks)Multimodal77.663.5−14.1
FrontierCode 2.0 - Pass RateCode46.432.3−14.1
LMArena Agent Arena Praise vs ComplaintAgents30.53.1−13.7
CritPt (avg@5)Science31.418.0−13.4
FrontierCode 2.0 - ScoreCode42.228.9−13.3
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine32.018.7−13.3
PLE (Physical-Engineering's Last Exam)Professional Work45.232.0−13.2
Design Arena Overall Code CategoriesCode13491270−13.1
ChessBench EloGames8571076+13.0
APEX-SWECode66.453.4−13.0
MBCT (SecureBio)Science58.846.1−12.7
o11y-bench 1.1 Pass^3Agents89.176.6−12.5
SpatialBench-Long V1 Clean RerunsScience36.524.4−12.1
Artificial Analysis Coding Agent Index v1.5Indexes68.456.3−12.1
Roboflow Vision Evals (September 2026 protocol)Multimodal83.871.9−11.9
MKA (Artificial Analysis)Professional Work52.040.1−11.8
Humanity's Last Exam (Text-Only)General QA55.043.1−11.8
MedXpertQA MM (200-question set)Health & Medicine80.769.0−11.7
HealthBenchHealth & Medicine65.954.2−11.7
ACCoRDGeneral QA85.774.0−11.6
SAFE-Teen 106 worst branch: Health Advice (under-18 prompt)Safety30.819.2+11.5
APEX-1 (Mercor, mean score)Professional Work53.965.4+11.5
MetagenomicsBench V1Science60.048.7−11.3
MetagenomicsBench V0Science60.048.7−11.3
GBENCH One-shot Coding GScoreGames67.456.3−11.1
SAGEMath51.840.8−11.0
SAFE-Teen 106 worst branch: Relationships (no prompt)Safety37.047.9−10.9
BrowseComp (Mercor 130-question subset)Search86.675.8−10.8
VCT v2 (SecureBio)Science52.242.0−10.2
BioSecBench-Function V1 Clean RerunsScience53.543.3−10.1
TxBench Oligo Discovery V1 (113 evaluations)Science51.941.9−10.0
TaxCalcBench TY25Finance & Accounting18.08.0−10.0
Artificial Analysis Progress Index v1Indexes351323−9.8
Artificial Analysis Intelligence Index v4.3.2Indexes56.046.4−9.6
SimpleQA Verified (Epoch anti-abstention)General QA46.556.0+9.5
DAYJOB: FinanceFinance & Accounting24.014.5−9.5
CursorBench 4.0Code55.546.3−9.2
APEX-Accounting 1.1Finance & Accounting64.555.3−9.2
RedlineBenchLegal40.349.5+9.2
MineBench — Bradley-Terry RatingGames19981848−9.1
Eval Connections One-Shot with TrapsGames76.085.0+9.0
WellnessBenchSafety57.048.0−9.0
WCB (SecureBio)Science66.557.6−8.9
Excel Modeling BenchmarkFinance & Accounting75.767.0−8.7
GDP.pdfProfessional Work31.422.8−8.6
LMArena Agent Arena Steering BurdenAgents14.5-2.3−8.4
LiveBench 2.0Agents82.574.1−8.4
Artificial Analysis Economics IndexIndexes54.145.7−8.4
Epoch Capabilities Index (ECI)Indexes166156−8.3
EpiBench V1 Clean RerunsScience28.920.8−8.2
HLAB-Vals-1.1Legal10.418.3+7.9
DualEntry Accounting AI BenchmarkFinance & Accounting80.288.1+7.9
TxBench Preclinical Pharmacology V1 Clean RerunsScience62.354.7−7.7
MMMU ProMultimodal85.878.2−7.6
Artificial Analysis Healthcare & Medical IndexIndexes54.246.6−7.6
ATLAS (Exa, 200-task subset)Search46.839.5−7.3
LMArena Agent Arena Confirmed SuccessAgents19.55.1−7.2
SAFE-Teen 106: Urgent Safety (under-18 prompt)Safety2.910.1−7.2
Blueprint-Bench 2Multimodal0.40.3−6.8
GDP.xlsxProfessional Work29.122.3−6.8
DAYJOB: HealthcareHealth & Medicine15.28.4−6.8
SWE-bench MultilingualCode90.984.2−6.7
ComplexConstraints (November 2026 rescoring)Chat & Writing67.260.8−6.4
DAXBench v2Code86.292.6+6.4
ARC-AGI-1 Semi-PrivateGeneral QA96.590.2−6.3
Vibe Code Bench v1.1Code92.486.2−6.2
AutomationBench-AA 1.0.6Agents71.865.6−6.2
HealthBench (length-adjusted, Artificial Analysis)Health & Medicine62.156.0−6.0
CharXiv Extended (Mercor held-out)Multimodal84.278.2−6.0
ProgramBenchCode6.50.5−6.0
Ruby LLM Benchmarks — Program FixerCode73.868.0−5.8
DAYJOB: TaxFinance & Accounting10.85.0−5.8
AA-Briefcase v1.1 EloAgents17401636−5.7
CVDP (517-problem set)Code60.654.9−5.7
VariantBench V1 Clean RerunsScience51.145.5−5.6
SWE-Marathon v1.1Code51.345.6−5.6
AA-LCR Extended (Mercor held-out)Search73.868.2−5.6
SpeciEval (Refusal-Aware Retries, Historical)Safety91.096.4+5.4
SpeciEval (Pooled English Runs)Safety91.096.4+5.4
SWE Atlas v1.1 - OverallCode67.562.1−5.4
Clipping Bench v1.0Games886384−5.4
LMArena Text Arena EloChat & Writing14721442−5.3
TxBench Antibody Discovery V1 (100 evaluations)Science57.352.0−5.3
GPQA-Bio (SecureBio)Science87.682.5−5.1
FrontierCode 1.1 Extended - ScoreCode64.459.4−5.1
Artificial Analysis Legal IndexIndexes52.247.2−5.0
SAFE-Teen 106: Health Advice (under-18 prompt)Safety16.912.0+4.9
FrontierCode 1.1 Extended - Pass RateCode69.965.2−4.6
Visa Agentic Commerce BenchmarkAgents64.460.0−4.4
Excel Modeling Benchmark v1.1Finance & Accounting75.971.5−4.4
GDPxProfessional Work8.23.9−4.3
MissionReportProfessional Work63.859.5−4.3
CharXiv (descriptive and reasoning combined)Multimodal96.192.0−4.1
Artificial Analysis Finance & Accounting IndexIndexes55.851.7−4.1
LMArena Agent Arena Bash RecoveryAgents15.07.2−3.9
Vending-Bench 2Games1116510537−3.8
WMDP-Bio (SecureBio)Science92.688.8−3.8
Extended NYT Connections 940 (Quadratic v1)Games80.576.8−3.7
HANDBOOK.md Agents v1.1Professional Work47.744.0−3.7
BioSimBench 1.0Science16.012.3−3.7
SAFE-Teen 106 worst branch: Health Advice (no prompt)Safety42.939.2+3.7
MedXpertQA MM Extended (Mercor held-out)Health & Medicine55.652.0−3.6
Vision Arena OverallMultimodal12631249−3.5
MedCodeHealth & Medicine52.949.6−3.4
SWE Atlas - Codebase QnACode50.347.0−3.3
SciCode 1.0.1 (Artificial Analysis)Code61.057.8−3.2
Artificial Analysis Strategy & Ops IndexIndexes58.154.8−3.2
AdvancedIFChat & Writing83.480.2−3.2
Veracode GenAI Code Security (Fall 2026)Cybersecurity72.069.0−3.0
FrontierMath ErdősMath2.90.0−2.9
RCM-Bench-1Health & Medicine91.188.2−2.9
DeepResearch Bench II (Mercor 130-task set)Search51.749.0−2.7
SWE-TogetherCode62.465.1+2.7
YC-BenchGames17413151666389−2.6
EnterpriseBench: CoreCraftProfessional Work70.868.2−2.6
AA-AnalystAgent (pass^5)Professional Work57.555.0−2.5
Political Science LLM BenchmarkProfessional Work68.966.6−2.4
Terminal-Bench 2.1Code87.385.0−2.3
SAFE-Teen 106 worst branch: Relationships (under-18 prompt)Safety7.35.1+2.2
MMLU ProGeneral QA91.088.8−2.2
PubMedQA (SecureBio)Health & Medicine81.379.2−2.1
SWE-bench Verified Extended (Mercor held-out)Code57.355.3−2.0
OTIS Mock AIME 2024-2025Math100.098.1−1.9
ChessBench AccuracyGames75.874.0−1.9
AA-PersonalityBench Expressiveness indexChat & Writing73.375.3+1.9
RuneBenchGames5.55.4−1.8
MedScribeHealth & Medicine91.189.4−1.7
LMArena Agent Arena Bash ReliabilityAgents12.79.3−1.7
AML BenchFinance & Accounting67.565.8−1.7
Ramp Accounting BenchFinance & Accounting42.544.1+1.6
Public Benefits Bench v1.1Professional Work67.265.6−1.6
DTBenchGeneral QA97.396.0−1.3
Access to Justice BenchmarkLegal74.973.7−1.3
GDPval-AA v2.1Professional Work17431717−1.1
Hermes BenchAgents73.574.4+0.9
Immigration BenchLegal69.068.0−0.9
SM BenchSafety85.486.3+0.9
Financial Coaching BenchFinance & Accounting74.974.1−0.9
ObviousBenchGeneral QA100.099.3−0.7
ScBench V1 Clean RerunsScience59.158.5−0.7
SAGE v1.1Math88.088.7+0.7
SAFE-Teen 106: Relationships (under-18 prompt)Safety1.71.0+0.6
DAYJOB: LawLegal27.027.6+0.6
SpatialBench V1 Clean RerunsScience70.169.6−0.6
SAFE-Teen 106: Urgent Safety (no prompt)Safety14.715.1−0.4
ForecastBench Preliminary Dataset Brier IndexForecasting61.160.7−0.4
AA-LCR v1.1Search79.780.0+0.3
AA-Omniscience IndexIndexes32.332.0−0.1
MMLU-Bio (SecureBio)Science91.090.9−0.1
LMArena Agent Arena Tool HallucinationAgents0.91.0+0.1
AIME 2025Math100.0100.00.0
Dual-Arm Sim (task completion)Agents0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
MazeBench ASCIIGames1.01.00.0
VulcanBench Frontier v4 (functional pass@1)Code100.0100.00.0

Claude Sonnet 5.5vsGrok 4.7normalised score marginsData Nov 15, 2026Data: Vals AI, Artificial Analysis, Mercor and 68 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…