Atlas

Compare Models

Claude Opus 5 vs GPT-6.1 Sol

GPT-6.1 Sol leads by 2.2 capability points (≈95% interval −0.3 to +4.7): statistically tied · P(higher) ≈ 96%

Wins 119 of 217 shared benchmarks (71 lost, 27 tied)

Capability vs. relative cost
Claude Opus 5GPT-6.1 SolClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Opus 5vsGPT-6.1 Solcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins71 – 119 · 217 shared, 27 tied

Comparing matching setups only (80 benchmarks excluded) · suites counted once
← Claude Opus 5GPT-6.1 Sol →

Claude Opus 5vsGPT-6.1 Solwins by categoryData Nov 15, 2026Data: Vals AI, Mercor, Artificial Analysis and 55 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6.1 Sol minus Claude Opus 5.

Concept Synth INDUCTION Challenge100 Round 1General QA24.090.0+66.0
Concept Synth INDUCTION Challenge64 Round 1General QA32.895.3+62.5
ActiveVisionMultimodal10.671.8+61.2
HPCT Refusal (CAIS)Safety100.042.0−58.0
Clipping Bench v1.0Games7345707+53.5
Machiavelli Deception (CAIS)Safety96.578.8+53.4
ChessBench EloGames8261562+43.8
SWE-bench Verified Extended (Mercor held-out)Code82.045.3−36.7
YC-BenchGames1911592936267−34.0
Terminal-Bench-Science 0.1Science28.658.1+29.5
Olam Social Poker EloGames15181543+27.2
MMMU-Pro Extended (Mercor held-out)Multimodal34.361.3+27.0
FrontierMath Tier 4 (v2) - PrivateMath73.2100.0+26.8
Terminal-Bench Science 0.1 (Vals)Science27.152.9+25.7
Medical Long Context Reasoning (MLCR-AA)Health & Medicine59.433.9−25.6
Chess Puzzles (task version 2.0)Games43.068.0+25.0
CrossSearch (pass^3)Search34.059.0+25.0
VIALSMultimodal24.449.1+24.7
Agent Red Teaming k=100 (CAIS)Safety43.318.8+24.5
EQ-Bench 4Chat & Writing13651272−24.2
BrowseComp Extended (Mercor held-out)Search44.067.7+23.7
ChessBench CoherenceGames63.887.5+23.7
TextQuests Average Cumulative HarmSafety22.629.3−22.7
Mystery Game PuzzlesGames59.080.0+21.0
LisanBench Path LengthGames154288736−20.8
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal20.00.0−20.0
Furniture Assembly BenchmarkMultimodal60.880.0+19.2
NoCoT-Bench NCRI 15.2General QA115149+18.6
EnterpriseBench: CoreCraftProfessional Work68.750.3−18.4
TextQuests Game ProgressGames53.371.4+18.1
TaxCalcBench TY25Finance & Accounting20.038.0+18.0
VoxelBench Text RatingGames23002550+17.8
Public Benefits Bench v1.1Professional Work76.959.3−17.6
SM BenchSafety85.869.0−16.8
Legal Research BenchLegal55.338.5−16.8
APEX-SWECode63.746.9−16.8
HieroglyphBenchMultimodal14.631.4+16.8
BuseyBench SVGMultimodal6.38.0+16.5
Humanity's Sixth SenseMultimodal30.546.6+16.1
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine40.024.0−16.0
EnigmaEvalGeneral QA43.959.5+15.6
MedCodeHealth & Medicine63.648.8−14.7
SpatialViz-BenchMultimodal75.890.3+14.5
Tax Agent BenchFinance & Accounting76.762.3−14.3
AutomationBench 1.0.6 (provider-reported)Agents50.336.1−14.2
Humanity's Last Exam DiamondGeneral QA44.858.8+14.0
LLM Chess EloGames12851612+13.4
VCT-Refusal (CAIS)Safety29.416.1−13.3
BioSecBench-Refusal V1 Clean RerunsSafety24.537.6+13.1
ALE-BenchCode26222990+13.0
IOI (Vals v2)Code84.396.9+12.6
Arena Alignment IndexSafety75.687.9+12.4
SpatialBench-Long V1 Clean RerunsScience28.039.9+11.9
Vals Science IndexIndexes47.959.6+11.7
LisanBench Difficulty-WeightedGames50684004−11.4
CritPt v2 (agentic, avg@5)Science71.482.6+11.1
GDP.pdfProfessional Work24.035.0+11.0
GeneBench-Pro Public Release SubsetScience22.033.0+11.0
GBENCH One-shot Coding GScoreGames68.179.0+10.9
BrowseComp (Mercor 130-question subset)Search84.695.4+10.8
Roboflow Vision Evals (October 2026, five tasks)Multimodal70.981.7+10.8
Artificial Analysis Intelligence Index v5.0.1Indexes44.254.6+10.4
ChessBench AccuracyGames71.080.7+10.4
DeepSWE v1.1Code65.075.2+10.2
SnorkelManufacturingProfessional Work9.419.6+10.2
HealthBenchHealth & Medicine67.156.9−10.2
TxBench Oligo Discovery V1 (113 evaluations)Science42.552.5+10.0
AutomationBench-AA 1.0.6Agents56.666.6+10.0
SnorkelFinance 2.0Finance & Accounting16.56.6−9.9
LMArena Agent Arena Net Improvement (Steering Burden)Agents7.212.5+9.9
FrontierCode 2.0 (Artificial Analysis)Code31.240.6+9.5
SpeciEval (Pooled English Runs)Safety87.997.4+9.4
SpeciEval (Refusal-Aware Retries, Historical)Safety87.997.4+9.4
BrokenArXiv 06/2026Math90.7100.0+9.3
MysteryMechanismScience37.446.4+9.0
ClockBench AccuracyMultimodal65.574.4+8.9
SWE Atlas Codebase QnA Extended (Mercor held-out)Code40.031.3−8.7
EBR-bench v4 (card ban, single agent)Games45.754.3+8.6
MetagenomicsBench V1Science54.045.7−8.3
Epiq LAER CorporateBenchProfessional Work74.082.2+8.2
FrontierMath Tiers 1-3 (v2) - PrivateMath85.693.7+8.1
BullshitBench v2 LeaderboardSafety73.065.0−8.0
Terminal-Bench 2.1 Extended (Mercor held-out)Code38.045.8+7.8
LMArena Agent Arena Praise vs ComplaintAgents12.227.5+7.7
Code MigrationCode57.565.1+7.6
ERQAMultimodal70.878.2+7.3
IntPhys 2Multimodal65.172.3+7.2
Roboflow Vision Evals (September 2026 protocol)Multimodal78.385.5+7.2
DeepsecBench — Revalidated V1 (233 issues)Code32.339.5+7.2
GameBench 2Games84.892.0+7.2
CVDP (517-problem set)Code62.255.1−7.1
NoCoT-Bench NCKI (kspine v3)General QA119130+7.1
Terminal-Bench 4.0 (Artificial Analysis)Code49.056.1+7.1
Search ArenaSearch12391230−6.9
MacroscopeBenchCode69.976.7+6.8
DAYJOB: FinanceFinance & Accounting11.318.0+6.7
SnorkelUnderwrite 2.0Finance & Accounting18.325.0+6.7
DAXBench v2Code86.292.6+6.4
MedXpertQA MM (200-question set)Health & Medicine79.885.7+5.9
Design Arena Overall Code CategoriesCode13211356+5.8
BioSecBench-Structure V1 (58 evaluations)Science15.521.3+5.7
SnorkelLegalLegal26.420.7−5.7
SWE-TogetherCode63.858.3−5.5
AA-Briefcase v1.1 EloAgents16541555−5.5
GDPval-AA v2.1Professional Work17291607−5.4
HiL-Bench (Human-in-Loop Benchmark)Agents57.062.3+5.3
ScBench V1 Clean RerunsScience58.363.6+5.3
CharXiv Extended (Mercor held-out)Multimodal79.885.1+5.3
Artificial Analysis Progress Index v1Indexes334348+5.2
Humanity's Last ExamGeneral QA56.451.3−5.1
HealthBench ProfessionalHealth & Medicine73.468.3−5.1
MASKSafety93.498.3+4.9
Ruby LLM Benchmarks — Program FixerCode79.875.0−4.8
MindCube TinyMultimodal84.389.1+4.8
MedScribeHealth & Medicine91.086.5−4.5
LMArena Agent Arena Bash RecoveryAgents13.64.5−4.5
VariantBench V1 Clean RerunsScience50.646.0−4.5
LMArena Agent Arena Steering BurdenAgents4.413.4+4.5
τ³-Banking (Fireworks, banking_knowledge)Agents49.144.7−4.4
FrontierMath ErdősMath0.04.4+4.4
Snorkel SWE-bench CLICode14.09.8−4.2
Toloka Arena (v3)Agents62.066.0+4.0
DistressBenchSafety70.274.2+4.0
Snorkel Agentic Coding 2.0Code38.942.8+3.9
APEX-Agents 1.1Professional Work77.073.2−3.8
AA-AnalystAgent (pass^5)Professional Work53.850.0−3.7
ARC-AGI-2General QA90.494.2+3.7
Terminal-Bench 2.1Code89.185.4−3.7
BioSecBench-Function V1 Clean RerunsScience42.546.0+3.6
Epoch Capabilities Index (ECI)Indexes164169+3.5
ARC-AGI-3 (Semi-Private)General QA30.226.7−3.4
TxBench Preclinical Pharmacology V1 Clean RerunsScience63.059.7−3.3
LiveBench 2.0Agents82.185.4+3.3
EpiBench V1 Clean RerunsScience25.228.3+3.1
SUPERChemScience74.777.8+3.1
ACCoRDGeneral QA83.880.9−3.0
SAGEMath49.446.5−2.9
BigFinanceBench private holdout (final-answer accuracy)Finance & Accounting49.252.0+2.8
Excel Modeling BenchmarkFinance & Accounting73.670.8−2.8
Agents' Last Exam Overall ScoreAgents55.958.6+2.7
Conceptual Reasoning IndexIndexes74.772.7−2.7
SciCode (main problems)Science17.314.6−2.7
Harvey LAB: In-House Contracting (held-out)Legal2.04.7+2.7
CritPt (avg@5)Science29.131.7+2.6
CheatBenchSafety45.342.8+2.5
HardGeoBench <=100miMultimodal46.343.8−2.5
HardGeoBench <=10miMultimodal32.535.0+2.5
LMArena Agent Arena Bash ReliabilityAgents9.85.0−2.4
SimpleBenchGeneral QA80.682.9+2.3
AA-Omniscience IndexIndexes37.141.5+2.2
KernelBench CUDA Normalized (RTX PRO 6000)Code45.543.2−2.2
BioSecBench-RespiratoryVirus-Surveillance V1 (110 evaluations)Science14.612.4−2.1
AA-LCR v1.1Search82.084.0+2.0
ProgramBench (Mercor 100-task subset)Code4.36.3+2.0
AdvancedIFChat & Writing80.582.4+1.9
Vision Arena OverallMultimodal12851293+1.8
LMCAGeneral QA64.562.8−1.7
CharXiv (descriptive and reasoning combined)Multimodal94.696.0+1.4
LMArena Text Arena EloChat & Writing14901482−1.3
DTBenchGeneral QA97.996.5−1.3
HardGeoBench Country AccuracyMultimodal53.852.5−1.2
OSWorld 2 (Aug 8) offline - reported partial reward, step limit unspecifiedAgents70.271.4+1.2
Extended NYT Connections 940 (Quadratic v1)Games94.395.5+1.2
SpatialBench V1 Clean RerunsScience68.467.3−1.2
OTIS Mock AIME 2024-2025Math98.9100.0+1.1
MMMU ProMultimodal84.085.1+1.1
Artificial Analysis Intelligence Index v4.3.2Indexes50.851.8+1.1
ARC-AGI-1 Semi-PrivateGeneral QA97.598.5+1.0
MMLU ProGeneral QA92.091.0−1.0
TxBench Antibody Discovery V1 (100 evaluations)Science48.749.7+1.0
τ³-Banking (Mercor run, bm25_grep retrieval)Agents44.043.0−1.0
DAYJOB: HealthcareHealth & Medicine8.47.6−0.8
LMArena Agent Arena Confirmed SuccessAgents10.211.7+0.8
Bedside BenchHealth & Medicine94.693.9−0.7
BioSecBench-Surveillance V1 Clean RerunsScience51.550.8−0.7
HardGeoBench Avg DistanceMultimodal12181239−0.6
MedXpertQA MM Extended (Mercor held-out)Health & Medicine64.164.7+0.6
PCT Political Inconsistency (CAIS)Safety36.737.3−0.6
SciCode 1.0.1 (Artificial Analysis)Code56.455.8−0.6
ScBench-Long V1 Clean RerunsScience27.326.7−0.6
Snorkel Digital Work IndexIndexes21.821.2−0.6
Terminal-Bench 4.0Code53.953.4−0.5
Vibe Code Bench v1.1Code88.488.9+0.5
Wiz Cyber Model Arena v3.9Cybersecurity71.270.7−0.5
BioMysteryBench (Vals Terminus 2)Science79.379.6+0.4
ForecastBench Preliminary Dataset Brier IndexForecasting62.061.7−0.3
Snorkel WorkplaceAgentsProfessional Work14.915.0+0.1
SnorkelRevOpsProfessional Work12.212.1−0.1
Harvey HLAB — Criteria Pass RateLegal87.787.8+0.1
LMArena Agent Arena Tool HallucinationAgents0.91.00.0
AIME 2025Math100.0100.00.0
APEX-Accounting 1.1Finance & Accounting64.364.30.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal20.020.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.00.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal20.020.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.00.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal20.020.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
ObviousBenchGeneral QA100.0100.00.0
ProofBench v1.1Math99.099.00.0

Claude Opus 5vsGPT-6.1 Solnormalised score marginsData Nov 15, 2026Data: Vals AI, Mercor, Artificial Analysis and 55 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…