Atlas

Compare Models

Opus 4.8 vs GPT-6.1 Sol

GPT-6.1 Sol leads by 6.6 capability points (≈95% interval +4.3 to +8.8): a clear lead over Opus 4.8 · P(higher) ≈ 100%

Wins 126 of 158 shared benchmarks (32 lost)

Capability vs. relative cost
Opus 4.8GPT-6.1 SolClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Opus 4.8vsGPT-6.1 Solcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins32 – 126 · 158 shared

Comparing matching setups only (60 benchmarks excluded) · suites counted once
← Opus 4.8GPT-6.1 Sol →

Opus 4.8vsGPT-6.1 Solwins by categoryData Nov 15, 2026Data: Vals AI, Mercor, Center for AI Safety and 46 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6.1 Sol minus Opus 4.8.

Concept Synth INDUCTION Challenge64 Round 1General QA6.395.3+89.1
Clocktower BenchGames11951764+81.6
ActiveVisionMultimodal2.471.8+69.4
BrokenArXiv 05/2026Math23.090.0+67.0
VoxelBench Text RatingGames16272550+65.7
BrokenArXiv 06/2026Math41.7100.0+58.3
Terminal-Bench Science 0.1 (Vals)Science4.352.9+48.6
LLM Chess EloGames4501612+47.7
Mystery Game PuzzlesGames36.080.0+44.0
FrontierMath Tier 4 (v2) - PrivateMath56.1100.0+43.9
HPCT Refusal (CAIS)Safety85.042.0−43.0
EnigmaEvalGeneral QA20.559.5+39.0
BrowseComp Extended (Mercor held-out)Search30.067.7+37.7
Furniture Assembly BenchmarkMultimodal42.580.0+37.5
ChessBench EloGames9611562+35.8
Humanity's Last Exam DiamondGeneral QA23.158.8+35.7
Terminal-Bench 4.0 (Artificial Analysis)Code21.756.1+34.3
DeepsecBench — Revalidated V1 (233 issues)Code6.939.5+32.6
Terminal-Bench 4.0Code23.255.1+31.8
ALE-BenchCode21022990+31.5
TextQuests Game ProgressGames40.171.4+31.3
ArXivMath 05/2026Math65.095.0+30.0
MMMU-Pro Extended (Mercor held-out)Multimodal31.461.3+29.9
GameBench 2Games62.592.0+29.5
BullshitBench v2 LeaderboardSafety94.065.0−29.0
TextQuests Average Cumulative HarmSafety21.029.3−28.3
Agent Red Teaming k=100 (CAIS)Safety47.018.8+28.2
BuseyBench SVGMultimodal5.28.0+27.4
Olam Social Poker EloGames15181543+27.2
YC-BenchGames1711389936267−27.1
HiL-Bench (Human-in-Loop Benchmark)Agents35.362.3+27.0
KernelBench Mega Normalized (RTX PRO 6000)Code40.614.1−26.5
NoCoT-Bench NCRI 15.2General QA102149+25.8
ARC-AGI-3 (Semi-Private)General QA1.526.7+25.2
VCT-Refusal (CAIS)Safety41.216.1−25.1
SpatialViz-BenchMultimodal65.290.3+25.0
GBENCH One-shot Coding GScoreGames54.079.0+25.0
ArXivMath 06/2026Math70.094.4+24.5
MindCube TinyMultimodal64.989.1+24.2
Roboflow Vision Evals (October 2026, five tasks)Multimodal58.381.7+23.4
Humanity's Sixth SenseMultimodal23.546.6+23.1
ARC-AGI-2General QA72.194.2+22.1
FACTS Benchmark SuiteGeneral QA53.575.3+21.8
BrowseComp (Mercor 130-question subset)Search73.795.4+21.7
AutomationBench-AA 1.0.6Agents45.666.6+21.0
SimpleQA Verified (Epoch anti-abstention)General QA53.073.9+20.9
DeepSWE v1.1Code54.475.2+20.9
ERQAMultimodal59.578.2+18.7
KernelBench CUDA Normalized (RTX PRO 6000)Code24.943.2+18.3
SimpleBenchGeneral QA64.882.9+18.1
SWE-bench Verified Extended (Mercor held-out)Code63.345.3−18.0
TaxCalcBench TY25Finance & Accounting20.038.0+18.0
Code MigrationCode47.365.1+17.9
SUPERChemScience60.377.8+17.5
HieroglyphBenchMultimodal14.031.4+17.4
Design Arena Overall Code CategoriesCode12521356+17.2
IntPhys 2Multimodal55.172.3+17.2
Roboflow Vision Evals (September 2026 protocol)Multimodal68.785.5+16.8
PCT Political Inconsistency (CAIS)Safety53.837.3+16.5
TxBench Antibody Discovery V1 (100 evaluations)Science33.349.7+16.3
Chess Puzzles (task version 2.0)Games52.068.0+16.0
TxBench Oligo Discovery V1 (113 evaluations)Science37.552.5+15.0
GeneBench-Pro Public Release SubsetScience18.033.0+15.0
Machiavelli Deception (CAIS)Safety83.778.8+14.9
LisanBench Difficulty-WeightedGames26944004+14.0
FrontierMath Tiers 1-3 (v2) - PrivateMath80.093.7+13.7
Arena Alignment IndexSafety74.587.9+13.4
Agents' Last Exam Overall ScoreAgents45.258.6+13.4
MedXpertQA MM (200-question set)Health & Medicine72.585.7+13.2
MedXpertQA MM Extended (Mercor held-out)Health & Medicine51.664.7+13.1
Terminal-Bench 2.1 Extended (Mercor held-out)Code32.745.8+13.1
SpeciEval (Pooled English Runs)Safety84.397.4+13.1
SpeciEval (Refusal-Aware Retries, Historical)Safety84.397.4+13.0
AA-Briefcase v1.1 EloAgents13201555+12.9
Medical Long Context Reasoning (MLCR-AA)Health & Medicine45.633.9−11.7
GDP.pdfProfessional Work24.035.0+11.0
Artificial Analysis Progress Index v1Indexes318348+10.6
ScBench-Long V1 Clean RerunsScience16.526.7+10.2
Artificial Analysis Intelligence Index v4.3.2Indexes41.851.8+10.0
ChessBench AccuracyGames72.480.7+8.9
APEX-Agents 1.1Professional Work64.473.2+8.8
Public Benefits Bench v1.1Professional Work68.159.3−8.8
AdvancedIFChat & Writing73.782.4+8.7
LiveBench 2.0Agents76.885.4+8.6
NoCoT-Bench NCKI (kspine v3)General QA117130+8.6
CharXiv Extended (Mercor held-out)Multimodal76.785.1+8.4
SAGEMath54.846.5−8.2
HealthBench ProfessionalHealth & Medicine60.368.3+8.0
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine16.024.0+8.0
ChessBench CoherenceGames79.787.5+7.8
HardGeoBench <=100miMultimodal36.343.8+7.5
HardGeoBench <=10miMultimodal27.535.0+7.5
SWE Atlas Codebase QnA Extended (Mercor held-out)Code24.031.3+7.3
Finance Agent v2 - Disclosure AnalysisFinance & Accounting65.758.4−7.3
Epoch Capabilities Index (ECI)Indexes160169+7.2
SpatialBench-Long V1 Clean RerunsScience33.339.9+6.6
ScBench V1 Clean RerunsScience57.163.6+6.5
GDPval-AA v2.1Professional Work14611607+6.4
AA-Omniscience IndexIndexes28.841.5+6.4
Vibe Code Bench v1.1Code82.788.9+6.2
Vals Index v2.1Indexes55.161.2+6.1
Conceptual Reasoning IndexIndexes68.372.7+6.0
ARC-AGI-1 Semi-PrivateGeneral QA92.598.5+6.0
Wiz Cyber Model Arena v3.9Cybersecurity65.470.7+5.3
Legal Research BenchLegal43.838.5−5.3
LMCAGeneral QA57.562.8+5.3
AA-AnalystAgent (pass^5)Professional Work45.050.0+5.0
Extended NYT Connections 940 (Quadratic v1)Games91.195.5+4.4
MedCodeHealth & Medicine53.248.8−4.4
Harvey's Legal Agent BenchmarkLegal9.65.4−4.2
SWE Atlas - Codebase QnACode44.448.4+4.0
SM BenchSafety72.969.0−4.0
BioSecBench-Surveillance V1 Clean RerunsScience46.950.8+3.9
MASKSafety94.498.3+3.9
VariantBench V1 Clean RerunsScience42.446.0+3.7
Finance Agent v2 - AdjustmentsFinance & Accounting48.444.7−3.7
Finance Agent v2 - All PassFinance & Accounting43.539.9−3.6
ACCoRDGeneral QA77.480.9+3.5
Terminal-Bench 2.1Code82.085.4+3.4
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting66.463.2−3.2
SWE-TogetherCode61.558.3−3.2
APEX-SWECode43.946.9+3.0
BioSecBench-Function V1 Clean RerunsScience43.146.0+3.0
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting71.674.5+2.9
CharXiv (descriptive and reasoning combined)Multimodal93.296.0+2.8
Finance Agent v2 - Market AnalysisFinance & Accounting67.264.5−2.8
DistressBenchSafety71.474.2+2.8
MetagenomicsBench V1Science48.345.7−2.7
Tax Agent BenchFinance & Accounting64.962.3−2.6
EpiBench V1 Clean RerunsScience25.828.3+2.5
Ruby LLM Benchmarks — Program FixerCode72.675.0+2.4
SciCode (main problems)Science16.914.6−2.3
LisanBench Path LengthGames94638736−2.3
Finance Agent v2 - ComparablesFinance & Accounting44.942.6−2.2
EnterpriseBench: CoreCraftProfessional Work52.350.3−2.0
ProgramBenchCode1.03.0+2.0
TxBench Preclinical Pharmacology V1 Clean RerunsScience57.759.7+2.0
HealthBenchHealth & Medicine58.856.9−1.9
AIME 2025Math98.3100.0+1.7
Vision Arena OverallMultimodal12861293+1.7
OTIS Mock AIME 2024-2025Math98.3100.0+1.7
DTBenchGeneral QA94.996.5+1.6
Humanity's Last ExamGeneral QA49.851.3+1.5
SpatialBench V1 Clean RerunsScience65.867.3+1.5
Excel Modeling BenchmarkFinance & Accounting69.470.8+1.4
SciCode 1.0.1 (Artificial Analysis)Code54.455.8+1.4
LMArena Agent Arena Bash RecoveryAgents7.24.5−1.3
HardGeoBench Country AccuracyMultimodal53.852.5−1.2
MMLU ProGeneral QA90.091.0+1.0
MedScribeHealth & Medicine85.886.5+0.7
ObviousBenchGeneral QA99.3100.0+0.7
Gray Swan IPI Attack Success (k=15)Safety8.08.6−0.6
Finance Agent v2 - Earnings AnalysisFinance & Accounting72.171.5−0.6
Finance Agent v2 - Financial ModelingFinance & Accounting23.323.1−0.3
HardGeoBench Avg DistanceMultimodal12451239+0.2
Finance Agent v2 - PrecedentsFinance & Accounting25.625.8+0.2
ForecastBench Preliminary Dataset Brier IndexForecasting61.661.7+0.1
LMArena Text Arena EloChat & Writing14821482+0.1

Opus 4.8vsGPT-6.1 Solnormalised score marginsData Nov 15, 2026Data: Vals AI, Mercor, Center for AI Safety and 46 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…