Atlas

Compare Models

Claude Opus 5 vs GPT-6 Sol

Claude Opus 5 leads by 0.3 capability points (≈95% interval −2.0 to +2.7): statistically tied · P(higher) ≈ 60%

Wins 118 of 220 shared benchmarks (79 lost, 23 tied)

Capability vs. relative cost
Claude Opus 5GPT-6 SolClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Opus 5vsGPT-6 Solcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins118 – 79 · 220 shared, 23 tied

Comparing matching setups only (103 benchmarks excluded) · suites counted once
← Claude Opus 5GPT-6 Sol →

Claude Opus 5vsGPT-6 Solwins by categoryData Nov 15, 2026Data: Vals AI, Mercor, Artificial Analysis and 56 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6 Sol minus Claude Opus 5.

Search ArenaSearch12391174−51.7
Medical Long Context Reasoning (MLCR-AA)Health & Medicine59.416.1−43.3
SWE-bench Verified Extended (Mercor held-out)Code82.039.3−42.7
ARC-AGI-3 (Semi-Private)General QA30.20.9−29.3
ChessBench EloGames8261303+28.4
Legal Research BenchLegal55.328.8−26.4
Olam Social Poker EloGames15181542+26.1
EnterpriseBench: CoreCraftProfessional Work68.743.1−25.6
ChessBench CoherenceGames63.888.2+24.4
Tax Agent BenchFinance & Accounting76.753.0−23.6
LisanBench Path LengthGames154288222−22.4
LisanBench Difficulty-WeightedGames50683118−20.9
DeepsecBench-AA (Artificial Analysis)Cybersecurity25.946.3+20.5
Public Benefits Bench v1.1Professional Work76.956.6−20.3
CadQueryEval (Corrected Geometry)Code80.0100.0+20.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal20.00.0−20.0
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine40.020.0−20.0
SM BenchSafety85.866.0−19.8
Vending-Bench 2Games1118214428+19.7
BrowseComp Extended (Mercor held-out)Search44.063.2+19.2
BullshitBench v2 LeaderboardSafety73.054.0−19.0
HealthBenchHealth & Medicine67.148.2−18.9
APEX-SWECode63.745.0−18.7
MMMU-Pro Extended (Mercor held-out)Multimodal34.351.9+17.6
FrontierMath Tier 4 (v2) - PrivateMath73.290.0+16.8
MedCodeHealth & Medicine63.647.1−16.5
YC-BenchGames19115921441793−16.4
ProofBench v1.1Math99.083.0−16.0
Eval Connections One-Shot with TrapsGames74.089.0+15.0
ZeroBench (Main questions, pass@5)Multimodal26.041.0+15.0
ComplexConstraintsChat & Writing37.350.8+13.5
BioSecBench-Refusal V1 Clean RerunsSafety24.537.9+13.4
DeepResearch Bench II (Mercor 130-task set)Search56.142.9−13.2
Chess Puzzles (task version 2.0)Games43.056.0+13.0
dfbench Detection RecallCybersecurity45.457.7+12.3
BuseyBench SVGMultimodal6.37.5+12.3
HealthBench ProfessionalHealth & Medicine73.461.2−12.2
Bio Capabilities Index (SecureBio)Indexes178161−12.2
Arena Alignment IndexSafety75.687.6+12.0
MetagenomicsBench V0Science54.042.0−12.0
MetagenomicsBench V1Science54.042.0−12.0
VariantBench V1 Clean RerunsScience50.638.7−11.9
ZeroBench Main Mean pass@1 (five samples)Multimodal17.228.8+11.6
HieroglyphBenchMultimodal14.626.0+11.4
HPCT (SecureBio)Science78.667.8−10.8
LLM Chess EloGames12851545+10.7
SpeciEval (Pooled English Runs)Safety87.998.5+10.6
SpeciEval (Refusal-Aware Retries, Historical)Safety87.998.5+10.5
Terminal-Bench 4.0Code53.943.8−10.1
AA-Briefcase v1.1 EloAgents16541476−9.8
CVDP (517-problem set)Code62.252.7−9.5
GDPval-AA v2.1Professional Work17291514−9.5
CritPt v2 (agentic, avg@5)Science71.480.9+9.4
GlobeBenchGeneral QA15.925.0+9.1
MedScribeHealth & Medicine91.082.0−9.0
HardGeoBench <=100miMultimodal46.337.5−8.7
SWE Atlas Codebase QnA Extended (Mercor held-out)Code40.031.3−8.7
HardGeoBench Avg DistanceMultimodal12181497−8.6
DeepsecBench — Revalidated V1Code32.440.9+8.5
DeepsecBench — Revalidated V1 (233 issues)Code32.340.7+8.4
APEX-Agents 1.1Professional Work77.068.7−8.3
WCB (SecureBio)Science68.860.5−8.3
Humanity's Last Exam (Text-Only)General QA54.446.3−8.1
TaxCalcBench TY25Finance & Accounting20.028.0+8.0
EBR-bench v4 (card ban, single agent)Games45.753.3+7.6
HardGeoBench Country AccuracyMultimodal53.846.3−7.5
SimpleBenchGeneral QA80.673.1−7.5
MysteryMechanismScience37.430.2−7.2
ChessBench AccuracyGames71.077.4+6.9
VoxelBench Text RatingGames23002394+6.7
Harvey LAB held-out set (Fireworks run)Legal7.40.7−6.7
Bedside BenchHealth & Medicine94.688.1−6.5
Blueprint-Bench 2Multimodal0.30.4+6.5
AutomationBench 1.0.6 (Zapier)Agents26.933.2+6.3
HardGeoBench <=10miMultimodal32.526.3−6.2
GPQA-Bio (SecureBio)Science82.788.9+6.2
Roboflow Vision Evals (October 2026, five tasks)Multimodal70.976.9+6.0
ZeroBench Main Questions pass^5Multimodal11.017.0+6.0
OSWorld 2 (Aug 8) offline - reported partial reward, step limit unspecifiedAgents70.264.4−5.8
LMArena Text Arena EloChat & Writing14901457−5.8
Clipping Bench v1.0Games7341240+5.4
LMCAGeneral QA64.559.1−5.4
GameBench 2Games84.879.4−5.4
AutomationBench-AA 1.0.6Agents56.661.7+5.1
Terminal-Bench 4.0 (Artificial Analysis)Code49.043.9−5.1
APEX-1 (Mercor, mean score)Professional Work68.663.6−5.0
Epiq LAER CorporateBenchProfessional Work74.079.0+5.0
AA-Omniscience IndexIndexes37.127.1−5.0
Conceptual Reasoning IndexIndexes74.771.0−5.0
TxBench Antibody Discovery V1 Clean RerunsScience41.937.0−4.9
BrowseComp (Mercor 130-question subset)Search84.689.4+4.8
DAYJOB: HealthcareHealth & Medicine8.43.6−4.8
Design Arena Overall Code CategoriesCode13211293−4.6
SAGEMath49.444.8−4.6
BrokenArXiv 06/2026Math90.795.4+4.6
APEX-Accounting 1.1Finance & Accounting64.359.7−4.6
SciCode (main problems)Science17.312.7−4.6
BioMysteryBench (Vals Terminus 2)Science79.374.8−4.4
FrontierCode 2.0 (Artificial Analysis)Code31.235.6+4.4
Terminal-Bench 2.1 Extended (Mercor held-out)Code38.042.4+4.4
MacroscopeBenchCode69.974.3+4.4
FrontierMath Tiers 1-3 (v2) - PrivateMath85.689.8+4.2
Extended NYT Connections 940 (Quadratic v1)Games94.390.1−4.2
GDPval Extended (Mercor held-out rubric score)Professional Work62.958.7−4.2
Harvey HLAB — Criteria Pass RateLegal87.783.5−4.2
Toloka Arena (v3)Agents62.058.0−4.0
CharXiv Extended (Mercor held-out)Multimodal79.883.8+4.0
GeneBench-Pro Public Release SubsetScience22.026.0+4.0
Roboflow Vision Evals (September 2026 protocol)Multimodal78.382.3+4.0
MedXpertQA MM Extended (Mercor held-out)Health & Medicine64.168.0+3.9
BioSecBench-Surveillance V1 Clean RerunsScience51.547.7−3.7
AA-LCR Extended (Mercor held-out)Search73.970.2−3.7
Terminal-Bench 2.1Code89.185.4−3.7
GDP.pdf (Artificial Analysis, All-pass)Professional Work21.625.2+3.6
SpatialBench-Long V1 Clean RerunsScience28.031.6+3.6
Longform Creative WritingChat & Writing86.382.8−3.5
KernelBench CUDA Normalized (RTX PRO 6000)Code45.542.0−3.4
FORTRESS Average Risk Score (ARS)Safety16.813.5+3.4
LMArena Agent Arena Bash RecoveryAgents13.67.0−3.3
ScBench V1 Clean RerunsScience58.361.5+3.2
RiemannBenchMath68.064.8−3.2
SWE-TogetherCode63.860.6−3.2
BioSecBench-Function V1 Clean RerunsScience42.539.3−3.2
EpiBench V1 Clean RerunsScience25.222.0−3.1
Artificial Analysis Intelligence Index v4.3.2Indexes50.847.6−3.1
MMLU ProGeneral QA92.088.9−3.1
Mystery Game PuzzlesGames59.056.0−3.0
Vals Science IndexIndexes47.945.0−2.9
LMArena Agent Arena Steering BurdenAgents4.410.1+2.9
Terminal-Bench Science 0.1 (Vals)Science27.130.0+2.9
LMArena Agent Arena Confirmed SuccessAgents10.24.5−2.8
SAB RankingForecasting-0.1-0.1+2.7
BizAgentBench (strict success)Agents64.061.3−2.7
VCT (SecureBio)Science51.454.0+2.6
Furniture Assembly BenchmarkMultimodal60.858.3−2.5
LMArena Agent Arena Net ImprovementAgents9.58.8−2.4
DistressBenchSafety70.268.0−2.2
τ³-Banking (Fireworks, banking_knowledge)Agents49.147.0−2.1
BigFinanceBench private holdout (final-answer accuracy)Finance & Accounting49.247.1−2.1
GDPval (Mercor 213-task rubric score)Professional Work85.683.5−2.1
Excel Modeling BenchmarkFinance & Accounting73.671.5−2.0
ARC-AGI-1 Semi-PrivateGeneral QA97.595.5−2.0
DAYJOB: FinanceFinance & Accounting11.39.3−2.0
ProgramBench (Mercor 100-task subset)Code4.32.3−2.0
GBENCH One-shot Coding GScoreGames68.170.1+2.0
LMArena Agent Arena SteerabilityAgents10.714.5+1.9
SUPERChemScience74.772.8−1.9
Wiz Cyber Model Arena v3.9Cybersecurity71.269.3−1.9
BioSecBench-Structure V1 (58 evaluations)Science15.513.8−1.7
IOI (Vals v2)Code84.382.6−1.7
CritPt (avg@5)Science29.130.9+1.7
AA-LCR v1.1Search82.083.7+1.7
DeepSWE v1.1Code65.066.6+1.6
LAB-Bench CloningScenarios (SecureBio)Science100.098.5−1.5
MedXpertQA MM (200-question set)Health & Medicine79.881.3+1.5
FrontierMath ErdősMath0.01.5+1.5
MMMU ProMultimodal84.082.5−1.4
ACCoRDGeneral QA83.882.4−1.4
Terminal-Bench-Science 0.1Science28.630.0+1.4
SciCode 1.0.1 (Artificial Analysis)Code56.457.6+1.3
ScBench-Long V1 Clean RerunsScience27.328.4+1.1
OTIS Mock AIME 2024-2025Math98.9100.0+1.1
LMArena Agent Arena Praise vs ComplaintAgents12.210.0−1.1
Creative Writing v3 Rubric ScoreChat & Writing16.316.1−1.1
τ³-Banking (Mercor run, bm25_grep retrieval)Agents44.043.0−1.0
MBCT (SecureBio)Science60.261.1+0.9
SpatialBench V1 Clean RerunsScience68.467.5−0.9
ARC-AGI-2General QA90.489.6−0.8
Ruby LLM Benchmarks — Program FixerCode79.879.0−0.8
Humanity's Sixth SenseMultimodal30.531.2+0.7
ObviousBenchGeneral QA100.099.3−0.7
Harvey LAB: In-House Contracting (held-out)Legal2.01.3−0.7
SAB ResearchFinance & Accounting96.295.6−0.6
ForecastBench Preliminary Dataset Brier IndexForecasting62.062.6+0.6
LMArena Agent Arena Net Improvement (Steering Burden)Agents7.26.9−0.6
LMArena Agent Arena Bash ReliabilityAgents9.810.9+0.6
Vibe Code Bench v1.1Code88.487.8−0.6
TxBench Oligo Discovery V1 Clean RerunsScience40.040.6+0.6
DTBenchGeneral QA97.997.3−0.5
ALE-BenchCode26222637+0.5
Creative Writing v3 EloChat & Writing21352126−0.5
NoCoT-Bench NCKI (kspine v3)General QA119119−0.4
AdvancedIFChat & Writing80.580.9+0.4
CharXiv (descriptive and reasoning combined)Multimodal94.695.0+0.4
Humanity's Last Exam DiamondGeneral QA44.845.2+0.4
LAB-Bench LitQA2 (SecureBio)Science51.451.8+0.4
Epoch Capabilities Index (ECI)Indexes164164−0.4
MMLU-Bio (SecureBio)Science91.190.8−0.3
NoCoT-Bench NCRI 15.2General QA115115−0.3
Code MigrationCode57.557.2−0.3
LiveBench 2.0Agents82.181.9−0.2
Artificial Analysis Progress Index v1Indexes334334+0.1
LMArena Agent Arena Tool HallucinationAgents0.91.00.0
Artificial Analysis Intelligence Index v5.0.1Indexes44.244.20.0
AIME 2025Math100.0100.00.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.00.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.00.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
LAB-Bench ProtocolQA (SecureBio)Science81.181.10.0
TxBench Preclinical Pharmacology V1 Clean RerunsScience63.063.00.0

Claude Opus 5vsGPT-6 Solnormalised score marginsData Nov 15, 2026Data: Vals AI, Mercor, Artificial Analysis and 56 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…