Atlas

Compare Models

Claude Sonnet 5.5 vs GPT-6 Sol

Claude Sonnet 5.5 leads by 2.8 capability points (≈95% interval +0.4 to +5.3): a clear lead over GPT-6 Sol · P(higher) ≈ 99%

Wins 136 of 185 shared benchmarks (46 lost, 3 tied)

Capability vs. relative cost
Claude Sonnet 5.5GPT-6 SolClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Sonnet 5.5vsGPT-6 Solcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins136 – 46 · 185 shared, 3 tied

Comparing matching setups only (69 benchmarks excluded) · suites counted once
← Claude Sonnet 5.5GPT-6 Sol →

Claude Sonnet 5.5vsGPT-6 Solwins by categoryData Nov 15, 2026Data: Artificial Analysis, Mercor, benchmarks.bio and 52 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6 Sol minus Claude Sonnet 5.5.

KernelBench Mega Normalized (RTX PRO 6000)Code90.62.7−87.9
AA-PersonalityBench Informativeness indexChat & Writing84.810.5−75.0
AA-PersonalityBench Organization indexChat & Writing84.812.4−73.1
ARC-AGI-3 (Semi-Private)General QA72.49.5−63.0
AA-PersonalityBench Transparency indexChat & Writing10039.1−61.5
Medical Long Context Reasoning (MLCR-AA)Health & Medicine75.016.1−58.9
AA-PersonalityBench Imagination indexChat & Writing84.829.6−55.8
KernelBench CUDA Normalized (RTX PRO 6000)Code87.042.0−45.0
AA-PersonalityBench Expressiveness indexChat & Writing73.333.4−40.4
EnterpriseBench: CoreCraftProfessional Work70.843.1−27.7
AA-PersonalityBench Slop indexChat & Writing27.71.0−26.9
ChessBench EloGames8571303+26.5
HardGeoBench Avg DistanceMultimodal6831497−25.1
ChessBench CoherenceGames64.688.2+23.6
Terminal-Bench-Science 0.1Science53.330.0−23.3
ChessBenchGames39.362.0+22.7
APEX-SWECode66.445.0−21.4
Vending-Bench 2Games1116514428+19.8
Terminal-Bench 4.0 (Artificial Analysis)Code63.643.9−19.7
SM BenchSafety85.466.0−19.5
AA-PersonalityBench Warmth indexChat & Writing73.354.3−19.2
MysteryMechanismScience49.130.2−18.9
AA-Briefcase v1.1 EloAgents18161476−18.7
Terminal-Bench 4.0Code61.843.8−18.0
MetagenomicsBench V0Science60.042.0−18.0
MetagenomicsBench V1Science60.042.0−18.0
SWE-bench Verified Extended (Mercor held-out)Code57.339.3−18.0
HealthBenchHealth & Medicine65.948.2−17.6
Olam Social Poker EloGames15261542+17.4
LisanBench Path LengthGames137878222−17.3
Terminal-Bench Science 0.1 (Vals)Science47.130.0−17.1
LMArena Agent Arena Net Improvement (Steering Burden)Agents16.06.9−17.1
ProofBench v1.1Math100.083.0−17.0
Furniture Assembly BenchmarkMultimodal75.058.3−16.7
SWE-bench VerifiedCode96.780.3−16.4
APEX-Agents 1.1Professional Work83.768.7−15.0
DAYJOB: FinanceFinance & Accounting24.09.3−14.7
Decision-Bench v1.1Professional Work51.236.5−14.7
LLM Chess EloGames11951545+14.4
GDPval-AA v2.1Professional Work18381514−14.3
SimpleQA Verified (Epoch anti-abstention)General QA46.560.7+14.2
SWE Atlas Codebase QnA Extended (Mercor held-out)Code44.731.3−13.4
Eval Connections One-Shot with TrapsGames76.089.0+13.0
GeneBench-Pro Public Release SubsetScience39.026.0−13.0
VariantBench V1 Clean RerunsScience51.738.7−13.0
Code MigrationCode69.857.2−12.6
HardGeoBench <=100miMultimodal50.037.5−12.5
HardGeoBench Country AccuracyMultimodal58.846.3−12.5
BioSecBench-Function V1 Clean RerunsScience51.739.3−12.4
MedXpertQA MM Extended (Mercor held-out)Health & Medicine55.668.0+12.4
Financial Coaching BenchFinance & Accounting74.962.6−12.3
GDPval Extended (Mercor held-out rubric score)Professional Work70.958.7−12.2
GDP.xlsxProfessional Work29.117.1−12.0
GameBench 2Games91.479.4−12.0
DAYJOB: HealthcareHealth & Medicine15.23.6−11.6
TxBench Antibody Discovery V1 (100 evaluations)Science54.743.3−11.3
τ³-Banking (Mercor run, bm25_grep retrieval)Agents54.043.0−11.0
NoCoT-Bench NCRI 15.2General QA95.1115+10.7
Public Benefits Bench v1.1Professional Work67.256.6−10.6
AA-3D v1.0 EloCode12041131−10.5
YC-BenchGames17413151441793−10.5
Artificial Analysis Intelligence Index v5.0.1Indexes54.544.2−10.4
LMArena Agent Arena Praise vs ComplaintAgents30.510.0−10.3
Quant-Arena (iterative total score)Finance & Accounting40.951.1+10.2
AutomationBench-AA 1.0.6Agents71.861.7−10.1
TaxCalcBench TY25Finance & Accounting18.028.0+10.0
BuseyBench SVGMultimodal6.57.5+10.0
HardGeoBench <=10miMultimodal36.326.3−10.0
Bio Capabilities Index (SecureBio)Indexes175161−9.7
APEX-1 (Mercor, mean score)Professional Work53.963.6+9.7
Extended NYT Connections 940 (Quadratic v1)Games80.590.1+9.6
HealthBench ProfessionalHealth & Medicine70.861.2−9.6
FrontierMath Tier 4 (v2) - PrivateMath80.590.0+9.5
HPCT (SecureBio)Science77.267.8−9.4
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine29.320.0−9.3
Nimble Web ContextSearch67.276.5+9.3
Design Arena Overall Code CategoriesCode13491293−9.3
SWE-bench MultilingualCode90.981.7−9.2
LMCAGeneral QA68.359.1−9.2
MedScribeHealth & Medicine91.182.0−9.1
Mystery Game PuzzlesGames65.056.0−9.0
TxBench Oligo Discovery V1 (113 evaluations)Science51.943.1−8.9
DeepResearch Bench II (Mercor 130-task set)Search51.742.9−8.8
Humanity's Last Exam (Text-Only)General QA55.046.3−8.7
Artificial Analysis Intelligence Index v4.3.2Indexes56.047.6−8.4
Conceptual Reasoning IndexIndexes77.171.0−8.3
EpiBench V1 Clean RerunsScience30.222.0−8.2
Humanity's Last Exam DiamondGeneral QA53.345.2−8.1
NoCoT-Bench NCKI (kspine v3)General QA107119+8.1
BioSecBench-Structure V1 (58 evaluations)Science21.813.8−8.0
ChartographyMultimodal61.653.6−8.0
CVDP (517-problem set)Code60.652.7−7.9
HealthBench (length-adjusted, Artificial Analysis)Health & Medicine62.154.2−7.9
SpireClimbGames1.39.2+7.8
AA-PersonalityBench Conviction indexChat & Writing58.150.5−7.7
SpeciEval (Pooled English Runs)Safety91.098.5+7.5
SpeciEval (Refusal-Aware Retries, Historical)Safety91.098.5+7.5
LMArena Agent Arena Confirmed SuccessAgents19.54.5−7.5
LAB-Bench ProtocolQA (SecureBio)Science88.381.1−7.2
SAGEMath51.844.8−7.0
ALE-BenchCode24482637+6.7
DistressBenchSafety74.668.0−6.6
SWE Atlas - Codebase QnACode50.344.1−6.2
FrontierCode 2.0 (Artificial Analysis)Code41.735.6−6.1
Toloka Arena (v3)Agents64.058.0−6.0
BullshitBench v2 LeaderboardSafety60.054.0−6.0
WCB (SecureBio)Science66.560.5−6.0
MedCodeHealth & Medicine52.947.1−5.8
Artificial Analysis Progress Index v1Indexes351334−5.8
ARC-AGI-2General QA83.989.6+5.7
BigFinanceBench private holdout (final-answer accuracy)Finance & Accounting41.547.1+5.6
Ruby LLM Benchmarks — Program FixerCode73.879.0+5.2
SpatialBench V1 Clean RerunsScience72.867.5−5.2
CritPt v2 (agentic, avg@5)Science75.780.9+5.1
Qodo AI Code Review Benchmark (125 PRs)Code54.048.9−5.1
APEX-Accounting 1.1Finance & Accounting64.559.7−4.8
Vibe Code Bench v1.1Code92.487.8−4.6
ProgramBenchCode6.52.0−4.5
Excel Modeling BenchmarkFinance & Accounting75.771.5−4.2
LMArena Agent Arena Bash RecoveryAgents15.07.0−4.0
AA-LCR v1.1Search79.783.7+4.0
DualEntry Accounting AI BenchmarkFinance & Accounting80.276.2−4.0
LAB-Bench LitQA2 (SecureBio)Science55.851.8−4.0
DeepSWE v1.1Code70.566.6−3.9
Short Form Bench v1.0Games23552394+3.9
Clipping Bench v1.0Games8861240+3.8
AA-LCR Extended (Mercor held-out)Search73.870.2−3.6
SpatialBench-Long V1 Clean RerunsScience34.931.6−3.4
SciCode 1.0.1 (Artificial Analysis)Code61.057.6−3.4
ACCoRDGeneral QA85.782.4−3.3
MMMU ProMultimodal85.882.5−3.3
DeepsecBench — Revalidated V1 (233 issues)Code37.540.7+3.2
Terminal-Bench 2.1 Extended (Mercor held-out)Code45.542.4−3.1
GDPval (Mercor 213-task rubric score)Professional Work86.583.5−3.0
Chess Puzzles (task version 2.0)Games59.056.0−3.0
VCT v2 (SecureBio)Science52.249.3−2.9
BrowseComp (Mercor 130-question subset)Search86.689.4+2.8
SimpleBenchGeneral QA75.973.1−2.8
LMArena Text Arena EloChat & Writing14721457−2.7
GBENCH One-shot Coding GScoreGames67.470.1+2.7
WMDP-Bio (SecureBio)Science92.690.0−2.6
AA-Omniscience IndexIndexes32.327.1−2.6
AdvancedIFChat & Writing83.480.9−2.5
Blueprint-Bench 2Multimodal0.40.4−2.4
BioSimBench 1.0Science16.013.6−2.4
MBCT (SecureBio)Science58.861.1+2.3
LMArena Agent Arena Steering BurdenAgents14.510.1−2.2
MMLU ProGeneral QA91.088.9−2.1
Terminal-Bench 2.1Code87.385.4−1.9
ScBench-Long V1 Clean RerunsScience30.328.4−1.9
Epoch Capabilities Index (ECI)Indexes166164−1.9
SWE-TogetherCode62.460.6−1.8
ChessBench AccuracyGames75.877.4+1.7
ForecastBench Preliminary Dataset Brier IndexForecasting61.162.6+1.5
LAB-Bench CloningScenarios (SecureBio)Science97.098.5+1.5
Roboflow Vision Evals (September 2026 protocol)Multimodal83.882.3−1.5
VCT (SecureBio)Science52.554.0+1.5
LisanBench Difficulty-WeightedGames29833118+1.4
FrontierMath ErdősMath2.91.5−1.4
TxBench Preclinical Pharmacology V1 Clean RerunsScience64.363.0−1.3
GPQA-Bio (SecureBio)Science87.688.9+1.3
Harvey's Legal Agent BenchmarkLegal2.91.7−1.2
ScBench V1 Clean RerunsScience60.361.5+1.2
CharXiv (descriptive and reasoning combined)Multimodal96.195.0−1.1
FrontierMath Tiers 1-3 (v2) - PrivateMath88.889.8+1.1
ARC-AGI-1 Semi-PrivateGeneral QA96.595.5−1.0
Veracode GenAI Code Security (Fall 2026)Cybersecurity72.073.0+1.0
WellnessBenchSafety57.056.0−1.0
LMArena Agent Arena Bash ReliabilityAgents12.710.9−0.9
Harvey LAB-AA v1.1 (hallucination-gated)Legal2.83.6+0.8
Roboflow Vision Evals (October 2026, five tasks)Multimodal77.676.9−0.7
ObviousBenchGeneral QA100.099.3−0.7
GDP.pdf (Artificial Analysis, All-pass)Professional Work25.825.2−0.6
MedXpertQA MM (200-question set)Health & Medicine80.781.3+0.6
CritPt (avg@5)Science31.430.9−0.6
LiveBench 2.0Agents82.581.9−0.5
IOI (Vals v2)Code83.182.6−0.4
CharXiv Extended (Mercor held-out)Multimodal84.283.8−0.4
PubMedQA (SecureBio)Health & Medicine81.380.9−0.4
MMLU-Bio (SecureBio)Science91.090.8−0.2
LMArena Agent Arena Tool HallucinationAgents0.91.0+0.1
Hermes BenchAgents73.573.50.0
AIME 2025Math100.0100.00.0
DTBenchGeneral QA97.397.30.0
OTIS Mock AIME 2024-2025Math100.0100.00.0

Claude Sonnet 5.5vsGPT-6 Solnormalised score marginsData Nov 15, 2026Data: Artificial Analysis, Mercor, benchmarks.bio and 52 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…