Atlas

Compare Models

Claude Sonnet 5.5 vs GPT-6.1 Sol

Claude Sonnet 5.5 leads by 0.3 capability points (≈95% interval −2.3 to +2.9): statistically tied · P(higher) ≈ 60%

Wins 111 of 222 shared benchmarks (105 lost, 6 tied)

Capability vs. relative cost
Claude Sonnet 5.5GPT-6.1 SolClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Sonnet 5.5vsGPT-6.1 Solcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins111 – 105 · 222 shared, 6 tied

Comparing matching setups only (92 benchmarks excluded) · suites counted once
← Claude Sonnet 5.5GPT-6.1 Sol →

Claude Sonnet 5.5vsGPT-6.1 Solwins by categoryData Nov 15, 2026Data: Artificial Analysis, Vals AI, Mercor and 59 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6.1 Sol minus Claude Sonnet 5.5.

KernelBench Mega Normalized (RTX PRO 6000)Code90.614.1−76.5
Short Form Bench v1.0Games23552949+58.9
AA-PersonalityBench Informativeness indexChat & Writing84.827.7−57.7
Clipping Bench v1.0Games8865707+51.9
Concept Synth INDUCTION Challenge100 Round 1General QA44.090.0+46.0
ViBench 2.0Code19.164.7+45.6
Concept Synth INDUCTION Challenge64 Round 1General QA50.095.3+45.3
KernelBench CUDA Normalized (RTX PRO 6000)Code87.043.2−43.8
ChessBench EloGames8571562+41.9
Medical Long Context Reasoning (MLCR-AA)Health & Medicine75.033.9−41.1
SAFE-Teen 106 worst branch: Health Advice (no prompt)Safety42.98.8+34.1
ActiveVisionMultimodal41.271.8+30.6
NoCoT-Bench NCRI 15.2General QA95.1149+29.6
ChessBenchGames39.368.7+29.3
SAFE-Teen 106: Health Advice (no prompt)Safety33.24.7+28.5
YC-BenchGames1741315936267−28.1
SimpleQA Verified (Epoch anti-abstention)General QA46.573.9+27.4
ARC-AGI-3 (Semi-Private)General QA72.496.4+23.9
AA-PersonalityBench Expressiveness indexChat & Writing73.350.5−23.1
AA-PersonalityBench Imagination indexChat & Writing84.861.9−23.1
SAFE-Teen 106 worst branch: Health Advice (under-18 prompt)Safety30.87.7+23.1
ChessBench CoherenceGames64.687.5+22.9
ATLAS (Exa, 200-task subset)Search46.869.5+22.7
SpireClimbGames1.323.8+22.5
EnterpriseBench: CoreCraftProfessional Work70.850.3−20.5
Agent Initiative BenchmarkAgents6.78.7+20.0
TaxCalcBench TY25Finance & Accounting18.038.0+20.0
FrontierMath Tier 4 (v2) - PrivateMath80.5100.0+19.5
APEX-SWECode66.446.9−19.5
ALE-BenchCode24482990+19.2
HANDBOOK.md Agents v1.1Professional Work47.728.6−19.1
Olam Social Poker EloGames15261543+18.5
HoneyBench v0.2 (hack rate)Safety41.659.5−17.9
SAFE-Teen 106 worst branch: Relationships (no prompt)Safety37.019.1+17.8
HardGeoBench Avg DistanceMultimodal6831239−17.1
LLM Chess EloGames11951612+17.1
sudo L7 (October 2026 rescoring)Code50.633.9−16.7
DAYJOB: Data ScienceProfessional Work35.318.7−16.6
SM BenchSafety85.469.0−16.5
LisanBench Path LengthGames137878736−15.7
NoCoT-Bench NCKI (kspine v3)General QA107130+15.6
Extended NYT Connections 940 (Quadratic v1)Games80.595.5+15.0
Mystery Game PuzzlesGames65.080.0+15.0
SWE-bench Multimodal (Vals v1, 480 Tasks)Code58.343.5−14.8
MetagenomicsBench V1Science60.045.7−14.3
AA-Briefcase v1.1 EloAgents18161555−14.3
BuseyBench SVGMultimodal6.58.0+14.2
IOI (Vals v2)Code83.196.9+13.8
SAFE-Teen 106: Health Advice (under-18 prompt)Safety16.93.3+13.6
AA-PersonalityBench Slop indexChat & Writing27.714.3−13.5
SWE Atlas Codebase QnA Extended (Mercor held-out)Code44.731.3−13.4
SWE-bench Verified Extended (Mercor held-out)Code57.345.3−12.0
AA-3D v1.0 EloCode12041121−12.0
GBENCH One-shot Coding GScoreGames67.479.0+11.6
SAFE-Teen 106: Urgent Safety (no prompt)Safety14.73.4+11.3
τ³-Banking (Mercor run, bm25_grep retrieval)Agents54.043.0−11.0
LisanBench Difficulty-WeightedGames29834004+10.9
APEX-Agents 1.1Professional Work83.773.2−10.5
BigFinanceBench private holdout (final-answer accuracy)Finance & Accounting41.552.0+10.5
ARC-AGI-2General QA83.994.2+10.3
GDPval-AA v2.1Professional Work18381607−10.2
DAYJOB: AccountingFinance & Accounting35.225.2−10.0
Code Migration v1.1Code86.977.3−9.6
SWE-bench VerifiedCode96.787.3−9.4
Access to Justice BenchmarkLegal74.965.8−9.1
MedXpertQA MM Extended (Mercor held-out)Health & Medicine55.664.7+9.1
WellnessBenchSafety57.066.0+9.0
Chess Puzzles (task version 2.0)Games59.068.0+9.0
HealthBenchHealth & Medicine65.956.9−8.9
TxBench-Antibody-Development V1 (60 evaluations)Science36.727.8−8.9
BrowseComp (Mercor 130-question subset)Search86.695.4+8.8
Terminal-Bench 4.0Code61.853.4−8.4
VIALSMultimodal41.049.1+8.1
SAFE-Teen 106 worst branch: Urgent Safety (no prompt)Safety23.215.4+7.9
Public Benefits Bench v1.1Professional Work67.259.3−7.8
AA-PersonalityBench Transparency indexChat & Writing10092.4−7.7
AA-PersonalityBench Warmth indexChat & Writing73.365.7−7.7
Vision Arena OverallMultimodal12631293+7.6
AIM GroundingProfessional Work86.694.2+7.6
DAYJOB: HealthcareHealth & Medicine15.27.6−7.6
Terminal-Bench 4.0 (Artificial Analysis)Code63.656.1−7.6
AA-AnalystAgent (pass^5)Professional Work57.550.0−7.5
GDP.xlsxProfessional Work29.122.0−7.1
SimpleBenchGeneral QA75.982.9+7.0
CritPt v2 (agentic, avg@5)Science75.782.6+6.9
DAYJOB: TaxFinance & Accounting10.84.0−6.8
SAFE-Teen 106: Relationships (no prompt)Safety16.910.2+6.7
LMArena Agent Arena Net Improvement (Steering Burden)Agents16.012.5−6.6
Financial Coaching BenchFinance & Accounting74.968.5−6.4
SpeciEval (Refusal-Aware Retries, Historical)Safety91.097.4+6.4
SpeciEval (Pooled English Runs)Safety91.097.4+6.4
DAXBench v2Code86.292.6+6.4
HardGeoBench <=100miMultimodal50.043.8−6.2
HardGeoBench Country AccuracyMultimodal58.852.5−6.2
Conceptual Reasoning IndexIndexes77.172.7−6.0
Code2Video-AA EloCode14431369−6.0
DAYJOB: FinanceFinance & Accounting24.018.0−6.0
GeneBench-Pro Public Release SubsetScience39.033.0−6.0
SWE-bench MultilingualCode90.984.9−6.0
Terminal-Bench Science 0.1 (Vals)Science47.152.9+5.7
BioSecBench-Function V1 Clean RerunsScience51.746.0−5.7
VariantBench V1 Clean RerunsScience51.746.0−5.6
Humanity's Last Exam DiamondGeneral QA53.358.8+5.5
CVDP (517-problem set)Code60.655.1−5.5
SpatialBench V1 Clean RerunsScience72.867.3−5.5
LMCAGeneral QA68.362.8−5.5
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine29.324.0−5.3
LMArena Agent Arena Bash RecoveryAgents15.04.5−5.3
ChessBench AccuracyGames75.880.7+5.3
SAGEMath51.846.5−5.2
SciCode 1.0.1 (Artificial Analysis)Code61.055.8−5.2
AutomationBench-AA 1.0.6Agents71.866.6−5.2
Humanity's Last ExamGeneral QA46.151.3+5.2
BullshitBench v2 LeaderboardSafety60.065.0+5.0
Furniture Assembly BenchmarkMultimodal75.080.0+5.0
MedXpertQA MM (200-question set)Health & Medicine80.785.7+5.0
TxBench Antibody Discovery V1 (100 evaluations)Science54.749.7−5.0
SpatialBench-Long V1 Clean RerunsScience34.939.9+5.0
Excel Modeling BenchmarkFinance & Accounting75.770.8−4.9
FrontierMath Tiers 1-3 (v2) - PrivateMath88.893.7+4.9
BioSecBench-RespiratoryVirus-Surveillance V1 (110 evaluations)Science17.312.4−4.8
Excel Modeling Benchmark v1.1Finance & Accounting75.971.1−4.8
DAYJOB: LawLegal27.031.8+4.8
ACCoRDGeneral QA85.780.9−4.8
Terminal-Bench-Science 0.1Science53.358.1+4.8
DeepSWE v1.1Code70.575.2+4.7
Code MigrationCode69.865.1−4.7
o11y-bench 1.1 Pass^3Agents90.685.9−4.7
TxBench Preclinical Pharmacology V1 Clean RerunsScience64.359.7−4.7
MedScribeHealth & Medicine91.186.5−4.6
AA-Omniscience IndexIndexes32.341.5+4.6
AA-LCR v1.1Search79.784.0+4.3
MKA (Artificial Analysis)Professional Work52.047.7−4.3
HLAB-Vals-1.1Legal10.46.3−4.2
Artificial Analysis Intelligence Index v4.3.2Indexes56.051.8−4.2
Roboflow Vision Evals (October 2026, five tasks)Multimodal77.681.7+4.1
SWE-TogetherCode62.458.3−4.1
MedCodeHealth & Medicine52.948.8−4.1
LMArena Agent Arena Confirmed SuccessAgents19.511.7−3.9
Ramp Accounting BenchFinance & Accounting42.546.4+3.9
LMArena Agent Arena Bash ReliabilityAgents12.75.0−3.9
AIM ConsistencyProfessional Work94.598.4+3.9
AA-PersonalityBench Conviction indexChat & Writing58.154.3−3.8
AA-PersonalityBench Organization indexChat & Writing84.881.0−3.8
GDP.pdfProfessional Work31.435.0+3.6
ScBench-Long V1 Clean RerunsScience30.326.7−3.6
HealthBench (length-adjusted, Artificial Analysis)Health & Medicine62.158.5−3.5
ProgramBenchCode6.53.0−3.5
Vibe Code Bench v1.1Code92.488.9−3.5
FrontierCode 1.1 Main (Artificial Analysis)Code45.849.1+3.3
ScBench V1 Clean RerunsScience60.363.6+3.3
LiveBench 2.0Agents82.585.4+3.0
ComplexConstraints (November 2026 rescoring)Chat & Writing67.270.1+2.9
SAFE-Teen 106 worst branch: Relationships (under-18 prompt)Safety7.34.5+2.8
MissionReportProfessional Work63.861.0−2.8
MysteryMechanismScience49.146.4−2.7
Harvey's Legal Agent BenchmarkLegal2.95.4+2.5
SAGE v1.1Math88.090.5+2.5
HealthBench ProfessionalHealth & Medicine70.868.3−2.5
RiemannBench (November 2026 rescoring)Math76.879.2+2.4
Artificial Analysis Economics IndexIndexes54.156.3+2.3
Visa Agentic Commerce BenchmarkAgents64.462.2−2.2
Artificial Analysis Healthcare & Medical IndexIndexes54.252.0−2.2
OpenWorldEngineer (Mean@8)Code39.041.2+2.2
Decision-Bench v1.1Professional Work51.249.1−2.1
Epoch Capabilities Index (ECI)Indexes166169+2.1
ARC-AGI-1 Semi-PrivateGeneral QA96.598.5+2.0
Toloka Arena (v3)Agents64.066.0+2.0
DeepsecBench — Revalidated V1 (233 issues)Code37.539.5+2.0
SWE Atlas - Codebase QnACode50.348.4−1.9
Terminal-Bench 2.1Code87.385.4−1.9
EpiBench V1 Clean RerunsScience30.228.3−1.9
Roboflow Vision Evals (September 2026 protocol)Multimodal83.885.5+1.7
LMArena Text Arena EloChat & Writing14721482+1.7
SWE Atlas v1.1 - OverallCode67.565.9−1.6
FrontierMath ErdősMath2.94.4+1.5
LMArena Agent Arena Praise vs ComplaintAgents30.527.5−1.5
Artificial Analysis Legal IndexIndexes52.253.7+1.5
HardGeoBench <=10miMultimodal36.335.0−1.2
Ruby LLM Benchmarks — Program FixerCode73.875.0+1.2
Design Arena Overall Code CategoriesCode13491356+1.2
Immigration BenchLegal69.067.8−1.2
FrontierCode 2.0 (Artificial Analysis)Code41.740.6−1.1
AdvancedIFChat & Writing83.482.4−1.0
DualEntry Accounting AI BenchmarkFinance & Accounting80.281.2+1.0
ProofBench v1.1Math100.099.0−1.0
SAFE-Teen 106: Urgent Safety (under-18 prompt)Safety2.92.0+0.9
CharXiv Extended (Mercor held-out)Multimodal84.285.1+0.9
SAFE-Teen 106 worst branch: Urgent Safety (under-18 prompt)Safety11.812.7−0.9
Artificial Analysis Engineering IndexIndexes58.559.4+0.9
DTBenchGeneral QA97.396.5−0.8
SAFE-Teen 106: Relationships (under-18 prompt)Safety1.70.9+0.8
Artificial Analysis Progress Index v1Indexes351348−0.7
MMMU ProMultimodal85.885.1−0.7
BioSimBench 1.0Science16.015.3−0.7
ForecastBench Preliminary Dataset Brier IndexForecasting61.161.7+0.6
TxBench Oligo Discovery V1 (113 evaluations)Science51.952.5+0.6
Political Science LLM BenchmarkProfessional Work68.968.3−0.6
BioSecBench-Structure V1 (58 evaluations)Science21.821.3−0.6
LMArena Agent Arena Steering BurdenAgents14.513.4−0.6
GameBench 2Games91.492.0+0.5
Artificial Analysis Finance & Accounting IndexIndexes55.856.2+0.5
Hermes BenchAgents73.573.9+0.5
Artificial Analysis Strategy & Ops IndexIndexes58.157.6−0.5
Prophet Arena Arena Score (Fixed Context)Forecasting85.384.8−0.5
DistressBenchSafety74.674.2−0.4
SWE-Bench Pro V2 — FullCode95.896.1+0.3
Prophet Arena Arena Score (Agentic, OpenCode)Forecasting85.786.0+0.3
Terminal-Bench 2.1 Extended (Mercor held-out)Code45.545.8+0.3
CritPt (avg@5)Science31.431.7+0.3
Artificial Analysis Intelligence Index v5.0Indexes54.254.5+0.3
GDPxProfessional Work8.28.4+0.2
APEX-Accounting 1.1Finance & Accounting64.564.3−0.2
CharXiv (descriptive and reasoning combined)Multimodal96.196.0−0.1
Artificial Analysis Intelligence Index v5.0.1Indexes54.554.6+0.1
LMArena Agent Arena Tool HallucinationAgents0.91.0+0.1
AIME 2025Math100.0100.00.0
ClaimBenchFinance & Accounting100.0100.00.0
Dual-Arm Sim (task completion)Agents0.00.00.0
MMLU ProGeneral QA91.091.00.0
ObviousBenchGeneral QA100.0100.00.0
OTIS Mock AIME 2024-2025Math100.0100.00.0

Claude Sonnet 5.5vsGPT-6.1 Solnormalised score marginsData Nov 15, 2026Data: Artificial Analysis, Vals AI, Mercor and 59 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…