Atlas

Compare Models

Opus 4.8 vs GPT-5.6 Sol

GPT-5.6 Sol leads by 3.1 capability points (≈95% interval +1.1 to +5.2): a clear lead over Opus 4.8 · P(higher) ≈ 100%

Wins 227 of 330 shared benchmarks (99 lost, 4 tied)

Capability vs. relative cost
Opus 4.8GPT-5.6 SolClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Opus 4.8vsGPT-5.6 Solcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins99 – 227 · 330 shared, 4 tied

Comparing matching setups only (146 benchmarks excluded) · suites counted once
← Opus 4.8GPT-5.6 Sol →

Opus 4.8vsGPT-5.6 Solwins by categoryData Nov 15, 2026Data: Vals AI, benchmarks.bio, Alibaba and 94 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-5.6 Sol minus Opus 4.8.

HPCT Refusal (CAIS)Safety85.024.0−61.0
Clocktower BenchGames11951583+55.7
Concept Synth INDUCTION Challenge64 Round 1General QA6.357.8+51.6
KernelBench Hard Normalized (H100)Code0.048.6+48.7
Opus Magnum CampaignGames22.870.9+48.0
VoxelBench Text RatingGames16272297+47.7
BullshitBench v2 LeaderboardSafety94.048.0−46.0
BridgeBench V3 Arena OverallIndexes1000934−45.7
KernelBench Mega Normalized (H100)Code63.818.6−45.2
LLM Chess EloGames4501550+45.1
SnakeBench Win RateGames60.0100.0+40.0
BabyVisionMultimodal28.465.5+37.1
TextQuests Average Cumulative HarmSafety21.031.4−35.2
YC-BenchGames1711389728001−34.3
MLVU (multiple-choice M-Avg)Multimodal53.487.6+34.2
SimpleQA VerifiedGeneral QA39.569.7+30.2
VCT-Refusal (CAIS)Safety41.211.3−29.9
SnakeBench Best ApplesGames28.018.0−29.4
GSO Opt@1Code47.176.5+29.4
SnakeBench RatingGames35.424.6−29.1
DeepsecBenchCode6.935.6+28.7
DeepsecBench — Revalidated V1Code6.935.4+28.6
DeepsecBench — Revalidated V1 (233 issues)Code6.935.3+28.4
RiemannBenchMath47.274.4+27.2
Aikido Known-CVE Detection (Pass@3 Union)Cybersecurity61.588.5+26.9
FrontierMath Tier 4 (v2) - PrivateMath56.182.9+26.8
BrokenArXiv 06/2026Math41.767.3+25.6
Into the Unknown Integrity Node Pass RateScience65.388.7+23.5
BioTIER RefusalSafety94.771.9−22.9
Mystery Game PuzzlesGames36.058.0+22.0
ActiveVisionMultimodal2.423.5+21.1
ARC-AGI-2General QA72.192.5+20.4
CadQueryEval (Corrected Geometry)Code68.088.0+20.0
CadQueryEvalCode56.076.0+20.0
Medical Long Context Reasoning (MLCR-AA)Health & Medicine45.626.1−19.4
LLM Creative Story WritingChat & Writing0.82.5+19.3
Blueprint-Bench 2Multimodal0.10.3+19.1
Gray Swan IPI Attack Success (k=15)Safety8.027.0−19.0
Frontier-Bench v0.1Agents18.837.5+18.7
ChessBench — Historical Output RepairGames24.042.7+18.7
ArXivMath 06/2026Math70.088.5+18.6
GBA Eval OverallGames70.952.6−18.3
Terminal-Bench 4.0 (Artificial Analysis)Code21.739.9+18.2
ScBench-Long V0 VerifiedScience25.843.9+18.2
GameBench 2Games62.580.1+17.6
ScBench-LongScience20.638.1+17.5
Olam Social Poker EloGames15181534+17.4
MRCR v2 (8-needle, Context Arena) - 128K pointwiseGeneral QA75.292.4+17.3
MetagenomicsBench V1Science48.331.3−17.0
MetagenomicsBench V0Science48.331.3−17.0
Visual Acuity 4K Threshold RowMultimodal10.012.0+16.7
MCPMark-VerifiedAgents76.492.9+16.5
DeepSWE v1.1Code54.470.7+16.4
ReactBench v1Code36.552.5+16.1
Terminal-Bench Science 0.1 (Vals)Science4.320.0+15.7
SpatialViz-BenchMultimodal65.280.7+15.5
ChessBench EloGames9611218+15.3
AutoformBenchMath44.058.9+15.0
ComplexConstraintsChat & Writing35.650.5+14.9
SnakeBench Average ApplesGames15.118.0+14.8
LisanBench Path LengthGames94634699−14.8
Into the Unknown Direction Node Pass RateScience60.174.9+14.7
SpeciEval (Pooled English Runs)Safety84.399.0+14.7
SpeciEvalSafety84.399.0+14.7
SpeciEval (Refusal-Aware Retries, Historical)Safety84.399.0+14.7
Terminal-Bench 4.0Code23.237.9+14.6
SnakeBench Total ApplesGames22636.0−14.4
Furniture Assembly BenchmarkMultimodal42.556.7+14.2
A-FantasiaGeneral QA50.036.0+14.0
CRED v0.92Science85.099.0+14.0
AHK-EvalCode80.694.4+13.9
Terminal-Bench 2.1Code71.985.8+13.9
LisanBench Difficulty-WeightedGames26941403−13.8
GBENCH One-shot Coding GScoreGames54.067.7+13.6
HieroglyphBenchMultimodal14.027.6+13.6
EBR-bench v2Games25.238.6+13.3
HPCT (SecureBio)Science63.176.3+13.2
ScBench-Long V1 Clean RerunsScience16.529.6+13.1
NL2Repo - release-reported repository generation scoreCode69.756.8−12.9
GeneBench-ProScience16.028.7+12.7
PerceptionBenchMultimodal47.259.7+12.5
PCT Political Inconsistency (CAIS)Safety53.841.7+12.1
Time Horizon Index: KSPAgents11.823.8+12.0
TextQuests Game ProgressGames40.152.1+12.0
MindCube TinyMultimodal64.976.8+11.9
AutomationBench 1.0.6 (Zapier)Agents16.928.8+11.9
TxBench Antibody Discovery V0Science45.633.8−11.9
SM BenchSafety72.961.1−11.8
ERQAMultimodal59.571.2+11.7
WorldVQA AccuracyMultimodal33.945.1+11.2
TasteVal (compute multiplier)Science0.80.6−11.0
Diligence Stack Agent BenchFinance & Accounting77.388.3+11.0
ALE-BenchCode21022410+10.9
BioSecBench-RefusalSafety28.639.5+10.9
SUPERChemScience60.371.0+10.7
RedlineBenchLegal39.750.3+10.6
IFBench (Artificial Analysis avg@5)Chat & Writing62.272.7+10.4
Roboflow Vision EvalsMultimodal68.178.3+10.2
𝜏³-BankingAgents34.244.3+10.1
TaxCalcBench TY25Finance & Accounting20.030.0+10.0
MedXpertQA MMHealth & Medicine71.781.5+9.8
Video-MMMUMultimodal75.385.0+9.7
DrugDiscoveryBenchScience46.856.5+9.7
Arena Alignment IndexSafety74.584.2+9.6
ChessBench CoherenceGames79.789.1+9.4
τ²-bench (Telecom)Agents94.485.1−9.4
MRCR v2 (8-needle, Context Arena) - 128K AUCGeneral QA87.897.1+9.3
AA-Briefcase EloAgents13161475+9.2
MedCodeHealth & Medicine53.244.0−9.2
SLAKEHealth & Medicine75.985.1+9.2
FrontierMath Tiers 1-3 (v2) - PrivateMath80.089.1+9.1
SWE Atlas - Codebase QnACode44.453.5+9.1
MATH-VisionMultimodal86.795.8+9.1
BioMysteryBench - Human SolvableScience88.579.5−9.0
BuseyBench SVGMultimodal5.26.1+9.0
GraphWalks BFS - 1MGeneral QA68.177.1+9.0
IntPhys 2Multimodal55.163.9+8.8
Agents' Last Exam Overall ScoreAgents45.253.6+8.4
EQ-Bench 4Chat & Writing12681236−8.3
BankerToolBench v1.1Finance & Accounting20.729.0+8.3
SpatialBench V0 VerifiedScience66.474.5+8.1
ProofBenchMath69.077.0+8.0
ClawEval-MM - pass@3Agents73.381.2+7.9
OSWorld 2.0 - reported partial reward, patch and step limit unspecifiedAgents54.862.6+7.8
OSWorld 2.0 500-step Partial ScoreAgents54.862.6+7.8
DECK-BenchProfessional Work66.974.7+7.8
SWE-bench VerifiedCode88.696.2+7.6
Terminal-Bench HardCode58.365.9+7.6
LiveCodeBench (Vals Public) - MediumCode89.882.2−7.6
SpatialBench-Long V0 VerifiedScience28.836.4+7.6
BioSecBench-Function V1 Clean RerunsScience43.135.5−7.5
MMMU ProMultimodal75.683.0+7.4
Humanity's Last Exam DiamondGeneral QA23.130.4+7.3
RealWorldQA - release-reported visual perceptionMultimodal76.683.7+7.1
GDPval-AA v2 (normalized)Professional Work54.561.6+7.1
Harvey's Legal Agent BenchmarkLegal9.62.5−7.1
VariantBench V1 Clean RerunsScience42.435.3−7.1
Chess Puzzles (task version 2.0)Games52.059.0+7.0
GDPval-AA v2.1Professional Work14611619+6.9
DeepResearch Bench II (Mercor 130-task set)Search44.651.5+6.9
Artificial Analysis Agentic IndexIndexes47.254.0+6.8
GDP.pdfProfessional Work24.030.7+6.7
Toloka Arena (v1)Agents54.347.6−6.7
Agent Red Teaming k=100 (CAIS)Safety47.040.3+6.7
Humanity's Sixth SenseMultimodal23.530.0+6.5
BioSecBench-Function V0Science44.638.3−6.3
HardGeoBench Country AccuracyMultimodal53.847.5−6.2
Drone-BenchAgents82.376.1−6.2
EnterpriseBench: CoreCraftProfessional Work52.346.2−6.1
VCT (SecureBio)Science48.254.3+6.1
WeirdML (v2)Science82.988.8+5.9
MMStarMultimodal76.782.5+5.8
HardGeoBench Avg DistanceMultimodal12451429−5.7
GeoBenchMultimodal41283843−5.7
Code MigrationCode47.352.9+5.7
Surface Evolver BenchAgents87.593.1+5.6
Hemingway-benchChat & Writing10381057+5.3
PubMedQA (SecureBio)Health & Medicine75.981.1+5.2
Artificial Analysis Intelligence Index v4.3.2Indexes41.847.0+5.2
Ramp SWE-Bench (78 published tasks)Code78.283.3+5.1
SkillsBench with SkillsAgents59.254.1−5.1
ClawEval-MM - average of three attemptsAgents73.878.9+5.1
Artificial Analysis Intelligence Index v4.3Indexes42.047.1+5.1
ARC-AGI-1 Semi-PrivateGeneral QA92.597.5+5.0
HardGeoBench <=100miMultimodal36.341.3+5.0
LAB-Bench ProtocolQA (SecureBio)Science76.981.9+5.0
Document ArenaProfessional Work14751480+4.9
Chartography (with tools)Multimodal75.079.9+4.9
CursorBench 3.2Code62.367.2+4.9
TxBench Oligo Discovery V0Science39.434.5−4.9
GraphWalks BFS - 256KGeneral QA85.990.7+4.8
MBCT (SecureBio)Science56.461.1+4.7
Terminal-Bench 2.1 Extended (Mercor held-out)Code32.737.4+4.7
CorpFin v2 - Max Fitting ContextFinance & Accounting69.865.2−4.7
SWE-Bench Pro (Public Dataset)Code69.264.6−4.6
Artificial Analysis Progress Index v1Indexes318331+4.6
GPQA-Bio (SecureBio)Science80.985.3+4.4
Legal Research BenchLegal43.848.1+4.3
ChessBench AccuracyGames72.476.4+4.3
Humanity's Last ExamGeneral QA49.845.5−4.3
Senior SWE-BenchCode30.534.7+4.2
SpatialBench-Long V1 Clean RerunsScience33.337.5+4.2
Long-Horizon Terminal-Bench 1.0 (mean reward)Code49.245.1−4.1
APEX-SWECode43.948.0+4.1
LAB-Bench LitQA2 (SecureBio)Science48.752.8+4.1
CharXiv ReasoningMultimodal80.584.6+4.1
Artificial Analysis Coding IndexIndexes74.378.3+4.1
NoCoT-Bench NCRI 15.2General QA102109+4.1
BabaIsBench Stage 0: The IntroGames96.0100.0+4.0
OEIS Open Lite ($200)Math39.043.0+4.0
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting71.675.6+4.0
Bio Capabilities Index (SecureBio)Indexes164169+3.9
BrowseCompSearch88.592.2+3.7
MathVision - Qwen corrected annotationsMath87.190.8+3.7
SWE-TogetherCode61.557.8−3.7
TxBench Preclinical Pharmacology V1 Clean RerunsScience57.761.3+3.7
GlobeBenchGeneral QA20.023.7+3.6
PointerbenchMultimodal68.965.3−3.6
APEX-AccountingFinance & Accounting48.051.5+3.5
HealthBenchHealth & Medicine58.855.3−3.5
VCT v2 (SecureBio)Science45.949.4+3.5
Video-MME (with subtitles)Multimodal86.089.5+3.5
LegalBenchLegal83.687.0+3.4
AA-Omniscience IndexIndexes28.822.0−3.4
SciCode 1.0.1 (Artificial Analysis)Code54.457.8+3.4
LMArena Agent Arena Bash RecoveryAgents7.20.7−3.2
EpiBenchScience39.042.1+3.1
Machiavelli Deception (CAIS)Safety83.782.7+3.1
PMC-VQAHealth & Medicine59.262.3+3.1
Finance Agent v2 - Market AnalysisFinance & Accounting67.264.2−3.1
Tax Agent BenchFinance & Accounting64.968.0+3.0
GPQA DiamondGeneral QA91.994.9+3.0
Co-Design BenchCode58.161.1+3.0
HiL-Bench (Human-in-Loop Benchmark)Agents35.332.3−3.0
WordleBenchGames94.097.0+3.0
Excel Modeling BenchmarkFinance & Accounting69.472.3+3.0
Vals Index v2.1Indexes55.158.0+2.9
DiligenceBenchFinance & Accounting47.150.0+2.9
LAB-Bench CloningScenarios (SecureBio)Science95.998.8+2.9
PMPP-HardCode62.359.4−2.9
MMLU Pro - LawGeneral QA81.979.1−2.8
Vals Index v2Indexes60.963.7+2.8
CorpFin v2 - Exact PagesFinance & Accounting64.361.5−2.8
ObviousBenchGeneral QA99.396.5−2.8
Vals IndexIndexes70.473.1+2.8
GDPval-AA v2Professional Work15291585+2.7
ScBench V1 Clean RerunsScience57.159.8+2.7
WorldVQA ForceAnswerMultimodal39.141.8+2.7
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine16.018.7+2.7
Finance Agent v2 - Financial ModelingFinance & Accounting23.320.7−2.6
MortgageTax (Vals all 1,258 documents)Finance & Accounting69.967.3−2.6
AndroidWorld - release-reported mobile useAgents75.077.6+2.6
WMDP-Bio (SecureBio)Science92.389.7−2.6
Epoch Capabilities Index (ECI)Indexes160163+2.5
Ruby LLM Benchmarks — Program FixerCode72.675.2+2.5
InteractiveBench (task pass)Agents28.330.8+2.5
AA-AnalystAgent (pass^5)Professional Work45.047.5+2.5
HardGeoBench <=10miMultimodal27.530.0+2.5
SCI-AGENT-EXAMScience57.159.6+2.5
NoCoT-Bench NCKI (kspine v3)General QA117121+2.4
Finance Agent v2 - Disclosure AnalysisFinance & Accounting65.763.3−2.4
SpatialBench V1 Clean RerunsScience65.868.1+2.3
PG-LLM-50Science0.40.4+2.3
BioMysteryBench - Human DifficultScience42.444.7+2.3
BioSecBench-SurveillanceScience50.252.5+2.3
BrowseComp Extended (Mercor held-out)Search30.032.3+2.3
WCB (SecureBio)Science66.764.4−2.3
Finance Agent v2 - Earnings AnalysisFinance & Accounting72.174.4+2.3
Vibe Code Bench v1.1Code82.780.5−2.2
SAGEMath54.852.6−2.2
Finance Agent v2 - All PassFinance & Accounting43.541.4−2.2
DistressBenchSafety71.473.6+2.1
OmniDocBench v1.5Multimodal87.985.8−2.1
LLM Debate BenchmarkChat & Writing16311645+2.0
Extended NYT Connections 940-Puzzle SetGames94.196.1+2.0
MMVUMultimodal79.281.2+2.0
LMArena Agent Arena Net ImprovementAgents6.96.3−2.0
MMLU Pro - Computer ScienceGeneral QA92.790.7−2.0
MMLU Pro - PhilosophyGeneral QA86.688.4+1.8
FrontierFinance v3.1Finance & Accounting45.046.8+1.8
Vals Multimodal IndexIndexes70.972.6+1.8
OTIS Mock AIME 2024-2025Math98.3100.0+1.7
Public Benefits Bench v1.1Professional Work68.166.5−1.6
HANDBOOK.mdProfessional Work21.923.5+1.6
PWNBench v0.1Cybersecurity55.654.0−1.6
LMArena Agent Arena SteerabilityAgents9.66.5−1.5
MMLU Pro - OtherGeneral QA86.187.7+1.5
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting66.464.9−1.5
ACCoRDGeneral QA77.475.9−1.5
SEA-HELMGeneral QA75.677.0+1.4
A-Fantasia (Valid Attempts with Retries)Games34.736.0−1.3
ABC-Bench liquid handling (SecureBio)Science100.098.7−1.3
TxBench Antibody Discovery V1 Clean RerunsScience30.231.4+1.2
LingoQAMultimodal73.872.6−1.2
Wiz Cyber Model Arena v3.9Cybersecurity65.464.2−1.2
MMLU Pro - EconomicsGeneral QA91.890.6−1.2
Antidote: Everyday EditionSafety10301027−1.1
EnterpriseOps-Gym-AAAgents44.042.9−1.0
Finance Agent v2 (Vals Index subset)Finance & Accounting53.955.0+1.0
Finance Agent v2 (Vals Multimodal Index component)Finance & Accounting53.955.0+1.0
FrontierCode 1.1 Extended - ScoreCode59.660.6+1.0
MASKSafety94.495.4+1.0
ScreenSpot-ProMultimodal82.381.3−1.0
LMCAGeneral QA57.558.5+1.0
MMLU Pro - HealthGeneral QA85.184.1−1.0
LiveCodeBench (Vals Public) - EasyCode99.198.1−0.9
AutomationBench (Overall)Agents17.218.1+0.9
DeepSearchQA (F1 score)Search92.293.1+0.9
MMLU Pro - ChemistryGeneral QA92.892.0−0.9
TaxEval v2Finance & Accounting75.674.8−0.9
Finance Agent v2 - AdjustmentsFinance & Accounting48.449.2+0.8
DTBenchGeneral QA94.995.7+0.8
MMLU Pro - HistoryGeneral QA82.781.9−0.8
MMLU Pro - PhysicsGeneral QA93.592.8−0.8
MMLU Pro - PsychologyGeneral QA90.089.2−0.8
SimpleVQAMultimodal67.366.6−0.7
SciCode (main problems)Science16.916.2−0.7
MMLU Pro - BusinessGeneral QA92.693.3+0.6
ARC-AGI-3 (Semi-Private)General QA1.52.1+0.6
Conceptual Reasoning IndexIndexes68.368.7+0.6
MMLU Pro - MathMath95.995.3−0.6
Finance Agent v2 - PrecedentsFinance & Accounting25.626.2+0.6
LMArena Text Arena EloChat & Writing14821485+0.6
Bench to the Future 3 — September Panel — PooledForecasting0.10.1−0.5
Vision Arena OverallMultimodal12861288+0.5
MedScribeHealth & Medicine85.885.2−0.5
LiveBench 2.0Agents76.876.3−0.5
APEX-AgentsProfessional Work39.439.9+0.5
ProgramBenchCode1.01.5+0.5
MMLU Pro (Vals 5-Shot CoT)General QA89.689.1−0.5
CorpFin v2 - Shared Max ContextFinance & Accounting66.066.4+0.5
CorpFin v2 (Vals Index component)Finance & Accounting66.066.4+0.5
CorpFin v2 (Vals Multimodal Index component)Finance & Accounting66.066.4+0.5
Finance Agent v2 - ComparablesFinance & Accounting44.945.3+0.4
Bench to the Future 3 — Pooled ScoreForecasting0.10.1+0.4
OSWorld-VerifiedAgents83.483.0−0.4
RecreationBench - Qwen internal application recreationCode48.047.6−0.4
BioSecBench-Surveillance V1 Clean RerunsScience46.947.3+0.4
SkillsBench without SkillsAgents43.743.3−0.4
TxBench Oligo Discovery V1 Clean RerunsScience35.335.0−0.3
ForecastBench Preliminary Dataset Brier IndexForecasting61.661.4−0.2
WitnessBench v1.0Games22.823.0+0.2
MMLU-Bio (SecureBio)Science90.991.1+0.2
MMLU Pro - BiologyGeneral QA94.794.6−0.1
MMLU Pro - EngineeringGeneral QA87.787.8+0.1
ForecastBench Tournament Overall Brier IndexForecasting66.266.1−0.1
EpiBench V1 Clean RerunsScience25.825.80.0
Into the UnknownScience0.00.00.0
MCP Atlas public (Kimi K3 100-turn evaluation)Agents83.683.60.0
VariantBenchScience42.142.10.0

Opus 4.8vsGPT-5.6 Solnormalised score marginsData Nov 15, 2026Data: Vals AI, benchmarks.bio, Alibaba and 94 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…