Atlas

Compare Models

Claude Sonnet 5.5 vs GPT-6 Astra

GPT-6 Astra leads by 0.6 capability points (≈95% interval −2.0 to +3.3): statistically tied · P(higher) ≈ 68%

Wins 123 of 249 shared benchmarks (97 lost, 29 tied)

Capability vs. relative cost
Claude Sonnet 5.5GPT-6 AstraClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Sonnet 5.5vsGPT-6 Astracapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins97 – 123 · 249 shared, 29 tied

Comparing matching setups only (81 benchmarks excluded) · suites counted once
← Claude Sonnet 5.5GPT-6 Astra →

Claude Sonnet 5.5vsGPT-6 Astrawins by categoryData Nov 15, 2026Data: Vals AI, Artificial Analysis, Mercor and 64 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6 Astra minus Claude Sonnet 5.5.

Clipping Bench v1.0Games8869514+92.8
AA-PersonalityBench Informativeness indexChat & Writing84.829.6−55.8
ChessBench EloGames8571755+53.4
Concept Synth INDUCTION Challenge100 Round 1General QA44.094.0+50.0
Concept Synth INDUCTION Challenge64 Round 1General QA50.098.4+48.4
Short Form Bench v1.0Games23552803+44.4
ViBench 2.0Code19.160.3+41.2
Medical Long Context Reasoning (MLCR-AA)Health & Medicine75.035.0−40.0
ActiveVisionMultimodal41.276.5+35.3
NoCoT-Bench NCRI 15.2General QA95.1159+35.1
SpireClimbGames1.334.0+32.7
ChessBench CoherenceGames64.697.0+32.4
ChessBenchGames39.370.0+30.7
SimpleQA Verified (Epoch anti-abstention)General QA46.575.6+29.1
EnterpriseBench: CoreCraftProfessional Work70.842.6−28.2
Olam Social Poker EloGames15261550+26.1
TaxCalcBench TY25Finance & Accounting18.044.0+26.0
ARC-AGI-3 (Semi-Private)General QA72.498.4+26.0
ALE-BenchCode24483164+25.4
Agent Initiative BenchmarkAgents6.79.0+23.2
RedlineBenchLegal40.362.8+22.5
Harvey's Legal Agent Benchmark - Corporate M&ALegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal20.00.0−20.0
LisanBench Path LengthGames137877393−19.9
sudo L7 (October 2026 rescoring)Code50.631.1−19.5
Mystery Game PuzzlesGames65.084.0+19.0
Eval Connections One-Shot with TrapsGames76.094.0+18.0
APEX-SWECode66.448.6−17.8
Extended NYT Connections 940 (Quadratic v1)Games80.598.1+17.6
Harvey HLAB — Criteria Pass RateLegal70.087.5+17.5
LLM Chess EloGames11951614+17.2
FrontierMath Tier 4 (v2) - PrivateMath80.597.6+17.1
IOI (Vals v2)Code83.1100.0+16.9
NoCoT-Bench NCKI (kspine v3)General QA107131+16.6
HANDBOOK.md Agents v1.1Professional Work47.731.4−16.3
Terminal-Bench Science 0.1 (Vals)Science47.162.9+15.7
SM BenchSafety85.469.9−15.5
APEX-1 (Mercor, mean score)Professional Work53.969.3+15.4
AA-PersonalityBench Expressiveness indexChat & Writing73.358.1−15.4
AA-PersonalityBench Organization indexChat & Writing84.869.5−15.4
Chess Puzzles (task version 2.0)Games59.074.0+15.0
SWE-bench Multimodal (Vals v1, 480 Tasks)Code58.343.3−15.0
DAYJOB: Data ScienceProfessional Work35.321.3−14.0
AA-Briefcase v1.1 EloAgents18161569−13.6
AA-PersonalityBench Conviction indexChat & Writing58.144.8−13.5
BigFinanceBench private holdout (final-answer accuracy)Finance & Accounting41.554.6+13.1
GBENCH One-shot Coding GScoreGames67.479.2+11.9
AA-PersonalityBench Slop indexChat & Writing27.716.2−11.5
ARC-AGI-2General QA83.995.0+11.1
GDPval-AA v2.1Professional Work18381587−11.1
DAXBench v2Code86.296.8+10.6
Humanity's Last Exam DiamondGeneral QA53.363.8+10.5
Blueprint-Bench 2Multimodal0.40.5+10.4
Quant-Arena (iterative total score)Finance & Accounting40.951.3+10.4
MetagenomicsBench V1Science60.049.7−10.3
MetagenomicsBench V0Science60.049.7−10.3
DAYJOB: LawLegal27.036.9+9.9
ChessBench AccuracyGames75.885.0+9.9
ChartographyMultimodal61.671.0+9.4
HoneyBench v0.2 (hack rate)Safety41.650.8−9.1
BullshitBench v2 LeaderboardSafety60.069.0+9.0
HealthBenchHealth & Medicine65.956.9−8.9
VIALSMultimodal41.049.9+8.9
APEX-Agents 1.1Professional Work83.775.1−8.6
HPCT (SecureBio)Science77.269.0−8.2
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine29.321.3−8.0
BuseyBench SVGMultimodal6.57.3+7.9
SimpleBenchGeneral QA75.983.6+7.7
AA-PersonalityBench Imagination indexChat & Writing84.877.2−7.7
BrowseComp (Mercor 130-question subset)Search86.694.2+7.6
Bio Capabilities Index (SecureBio)Indexes175164−7.6
CritPt v2 (agentic, avg@5)Science75.783.1+7.4
Financial Coaching BenchFinance & Accounting74.967.5−7.4
Qodo AI Code Review Benchmark (125 PRs)Code54.046.6−7.4
τ³-Banking (Mercor run, bm25_grep retrieval)Agents54.046.7−7.3
WellnessBenchSafety57.064.0+7.0
HardGeoBench Avg DistanceMultimodal683908−6.9
GDPval Extended (Mercor held-out rubric score)Professional Work70.964.1−6.8
GDP.pdf (Artificial Analysis, All-pass)Professional Work25.832.2+6.4
Code Migration v1.1Code86.980.6−6.3
AA-AnalystAgent (pass^5)Professional Work57.551.3−6.2
o11y-bench 1.1 Pass^3Agents90.684.4−6.2
Ramp Accounting BenchFinance & Accounting42.548.7+6.2
GDP.xlsxProfessional Work29.122.9−6.2
MedXpertQA MM (200-question set)Health & Medicine80.786.8+6.1
DAYJOB: TaxFinance & Accounting10.84.8−6.0
VCT (SecureBio)Science52.558.5+6.0
Roboflow Vision Evals (October 2026, five tasks)Multimodal77.683.4+5.8
VCT v2 (SecureBio)Science52.258.0+5.8
AA-PersonalityBench Transparency indexChat & Writing10094.3−5.8
AA-Omniscience IndexIndexes32.343.7+5.7
CVDP (517-problem set)Code60.654.9−5.7
PLE (Physical-Engineering's Last Exam)Professional Work42.748.3+5.6
SpeciEval (Refusal-Aware Retries, Historical)Safety91.096.5+5.6
SpeciEval (Pooled English Runs)Safety91.096.5+5.6
Nimble Web ContextSearch67.272.6+5.4
SAGEMath51.846.4−5.4
Vision Arena OverallMultimodal12631283+5.3
SpatialBench V1 Clean RerunsScience72.867.5−5.2
OpenWorldEngineer (Mean@8)Code39.044.2+5.2
DAYJOB: AccountingFinance & Accounting35.230.0−5.2
MedXpertQA MM Extended (Mercor held-out)Health & Medicine55.660.8+5.2
AA-3D v1.0 EloCode12041168−5.1
MBCT (SecureBio)Science58.863.9+5.1
VariantBench V1 Clean RerunsScience51.746.6−5.1
MaxxBenchCode66.971.9+5.1
TxBench Preclinical Pharmacology V1 Clean RerunsScience64.359.3−5.0
Furniture Assembly BenchmarkMultimodal75.080.0+5.0
FrontierMath Tiers 1-3 (v2) - PrivateMath88.893.7+4.9
RiemannBench (November 2026 rescoring)Math76.881.6+4.8
LMArena Agent Arena Bash RecoveryAgents15.05.5−4.8
micro1 PharmaBenchHealth & Medicine11.416.1+4.7
SciCode 1.0.1 (Artificial Analysis)Code61.056.5−4.5
MedCodeHealth & Medicine52.948.5−4.4
Conceptual Reasoning IndexIndexes77.173.9−4.4
SpatialBench-Long V1 Clean RerunsScience34.939.3+4.4
FrontierSWE V2 Worst@5Code50.955.1+4.2
MysteryMechanismScience49.153.2+4.1
Excel Modeling BenchmarkFinance & Accounting75.771.7−4.0
ACCoRDGeneral QA85.781.7−4.0
BioSecBench-RespiratoryVirus-Surveillance V1 (110 evaluations)Science17.313.3−3.9
Terminal-Bench 4.0Code61.857.9−3.9
WCB (SecureBio)Science66.562.6−3.9
LMCAGeneral QA68.364.4−3.9
TxBench-Antibody-Development V1 (60 evaluations)Science36.732.8−3.9
HealthBench (length-adjusted, Artificial Analysis)Health & Medicine62.158.2−3.8
HardGeoBench <=10miMultimodal40.036.3−3.7
LiveBench 2.0Agents82.586.1+3.7
DAYJOB: HealthcareHealth & Medicine15.211.6−3.6
ComplexConstraints (November 2026 rescoring)Chat & Writing67.270.8+3.6
DeepSWE v1.1Code70.574.1+3.6
ScBench V1 Clean RerunsScience60.363.9+3.6
Artificial Analysis Legal IndexIndexes52.255.6+3.4
HLAB-Vals-1.1Legal10.47.1−3.3
Artificial Analysis Intelligence Index v4.3.2Indexes56.052.7−3.3
Artificial Analysis Engineering IndexIndexes58.561.8+3.3
AutomationBench-AA 1.0.6Agents71.868.5−3.3
Excel Modeling Benchmark v1.1Finance & Accounting75.972.7−3.2
MedScribeHealth & Medicine91.187.9−3.2
Artificial Analysis Economics IndexIndexes54.157.2+3.1
BioSecBench-Function V1 Clean RerunsScience51.748.6−3.1
ScBench-Long V1 Clean RerunsScience30.327.3−3.0
GeneBench-Pro Public Release SubsetScience39.036.0−3.0
LAB-Bench CloningScenarios (SecureBio)Science97.0100.0+3.0
OSWorld 2.1 offline - reported partial reward, step limit unspecifiedAgents83.980.9−3.0
Toloka Arena (v3)Agents64.067.0+3.0
GameBench 2Games91.494.3+2.9
Roboflow Vision Evals (September 2026 protocol)Multimodal83.886.6+2.8
LMArena Agent Arena Confirmed SuccessAgents19.514.0−2.7
Physical-Engineering's Last Exam (hidden test)Professional Work43.145.8+2.7
LMArena Agent Arena Net Improvement (Steering Burden)Agents16.014.5−2.7
Terminal-Bench 2.1Code87.389.9+2.6
Terminal-Bench 4.0 (Artificial Analysis)Code57.159.6+2.5
DAYJOB: FinanceFinance & Accounting24.021.5−2.5
Dual-Arm Sim (task completion)Agents0.02.5+2.5
SWE Atlas - Codebase QnACode50.347.8−2.5
LMArena Agent Arena Bash ReliabilityAgents12.77.8−2.4
Terminal-Bench-Science 0.1Science52.454.8+2.4
Artificial Analysis Intelligence Index v5.0.1Indexes54.556.9+2.3
Epoch Capabilities Index (ECI)Indexes166169+2.3
Humanity's Last ExamGeneral QA56.954.8−2.1
Code MigrationCode69.867.7−2.1
MissionReportProfessional Work63.861.8−2.0
ARC-AGI-1 Semi-PrivateGeneral QA96.598.5+2.0
AA-PersonalityBench Warmth indexChat & Writing73.375.3+1.9
SWE Atlas v1.1 - OverallCode67.569.4+1.9
EpiBench V1 Clean RerunsScience30.228.3−1.9
ForecastBench Preliminary Dataset Brier IndexForecasting61.162.9+1.8
SWE-TogetherCode62.460.6−1.8
WMDP-Bio (SecureBio)Science92.690.8−1.8
MKA (Artificial Analysis)Professional Work52.050.2−1.7
Artificial Analysis Finance & Accounting IndexIndexes55.857.4+1.6
DeepResearch Bench II (Mercor 130-task set)Search51.750.1−1.6
AdvancedIFChat & Writing83.482.0−1.4
LisanBench Difficulty-WeightedGames29833113+1.4
GDPxProfessional Work8.29.6+1.4
Code2Video-AA EloCode14431427−1.3
TxBench Antibody Discovery V1 (100 evaluations)Science54.756.0+1.3
FrontierSWE V2 Best@5Code71.773.0+1.3
Ruby LLM Benchmarks — Program FixerCode73.875.1+1.3
Immigration BenchLegal69.067.7−1.3
HardGeoBench Country AccuracyMultimodal58.860.0+1.3
HardGeoBench <=100miMultimodal50.051.3+1.2
HealthBench ProfessionalHealth & Medicine70.869.7−1.1
SWE-Bench Pro V2 — FullCode95.896.9+1.1
Political Science LLM BenchmarkProfessional Work68.967.9−1.1
Artificial Analysis Intelligence Index v5.0Indexes54.255.3+1.0
LAB-Bench LitQA2 (SecureBio)Science55.854.8−1.0
DualEntry Accounting AI BenchmarkFinance & Accounting80.281.2+1.0
LAB-Bench ProtocolQA (SecureBio)Science88.389.3+1.0
ProgramBenchCode6.55.5−1.0
ProofBench v1.1Math100.099.0−1.0
CharXiv Extended (Mercor held-out)Multimodal84.285.1+0.9
FrontierCode 2.0 (Artificial Analysis)Code41.742.4+0.7
MMMU ProMultimodal85.886.4+0.6
BioSecBench-Structure V1 (58 evaluations)Science21.821.3−0.6
LMArena Agent Arena Praise vs ComplaintAgents30.531.7+0.6
Prophet Arena Arena Score (Fixed Context)Forecasting85.384.7−0.6
MMLU ProGeneral QA91.091.5+0.5
Artificial Analysis Healthcare & Medical IndexIndexes54.253.7−0.5
Prophet Arena Arena Score (Agentic, OpenCode)Forecasting85.786.2+0.5
AA-LCR v1.1Search79.780.0+0.3
Artificial Analysis Progress Index v1Indexes351350−0.3
MMLU-Bio (SecureBio)Science91.091.3+0.3
CritPt (avg@5)Science31.431.7+0.3
Decision-Bench v1.1Professional Work51.251.5+0.3
LMArena Text Arena EloChat & Writing14721474+0.3
DistressBenchSafety74.674.3−0.2
Hermes BenchAgents73.573.7+0.2
APEX-Accounting 1.1Finance & Accounting64.564.7+0.2
GDPval (Mercor 213-task rubric score)Professional Work86.586.7+0.2
GPQA-Bio (SecureBio)Science87.687.4−0.2
LMArena Agent Arena Steering BurdenAgents14.514.8+0.1
DeepsecBench — Revalidated V1 (233 issues)Code37.537.6+0.1
BioSimBench 1.0Science16.015.9−0.1
PubMedQA (SecureBio)Health & Medicine81.381.2−0.1
Artificial Analysis Strategy & Ops IndexIndexes58.158.0−0.1
FrontierCode 1.1 Extended - ScoreCode64.464.5+0.1
LMArena Agent Arena Tool HallucinationAgents0.91.0+0.1
FrontierMath ErdősMath2.92.90.0
AIME 2025Math100.0100.00.0
CharXiv (descriptive and reasoning combined)Multimodal96.196.10.0
DTBenchGeneral QA97.397.30.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.00.00.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal0.00.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal20.020.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
ObviousBenchGeneral QA100.0100.00.0
OTIS Mock AIME 2024-2025Math100.0100.00.0
TxBench Oligo Discovery V1 (113 evaluations)Science51.951.90.0
Visa Agentic Commerce BenchmarkAgents64.464.40.0

Claude Sonnet 5.5vsGPT-6 Astranormalised score marginsData Nov 15, 2026Data: Vals AI, Artificial Analysis, Mercor and 64 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…