Atlas

Compare Models

Opus 4.8 vs Claude Opus 5

Claude Opus 5 leads by 4.3 capability points (≈95% interval +2.2 to +6.5): a clear lead over Opus 4.8 · P(higher) ≈ 100%

Wins 258 of 313 shared benchmarks (30 lost, 25 tied)

Capability vs. relative cost
Opus 4.8Claude Opus 5Closed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Opus 4.8vsClaude Opus 5capability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins30 – 258 · 313 shared, 25 tied

Comparing matching setups only (34 benchmarks excluded) · suites counted once
← Opus 4.8Claude Opus 5 →

Opus 4.8vsClaude Opus 5wins by categoryData Nov 15, 2026Data: Vals AI, Mercor, benchmarks.bio and 88 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Claude Opus 5 minus Opus 4.8.

KernelBench Hard Normalized (H100)Code0.082.8+82.8
BrokenArXiv 06/2026Math41.790.7+49.1
VoxelBench Text RatingGames16272300+47.9
Firefox 147 Exploit DevelopmentCybersecurity8.852.4+43.6
Machiavelli Deception (CAIS)Safety83.796.5−38.5
KernelBench Mega Normalized (H100)Code63.8100.0+36.2
LLM Chess EloGames4501285+34.3
LLM Creative Story WritingChat & Writing0.83.7+33.6
Document ArenaProfessional Work14751509+30.5
Terminal-Bench 4.0Code23.653.9+30.3
ExploitBench - release-reported exploit progressCybersecurity40.070.0+30.0
ExploitBench Anthropic capability coverage, three-trial unionCybersecurity40.070.0+30.0
ARC-AGI-3 (Semi-Private)General QA1.530.2+28.6
ExploitBench Leaderboard (Mean capability score, out of 16)Cybersecurity5.610.1+28.6
Terminal-Bench 4.0 (Artificial Analysis)Code21.749.0+27.3
Concept Synth INDUCTION Challenge64 Round 1General QA6.332.8+26.6
DeepsecBenchCode6.932.6+25.7
DeepsecBench — Revalidated V1Code6.932.4+25.6
DeepsecBench — Revalidated V1 (233 issues)Code6.932.3+25.5
LisanBench Difficulty-WeightedGames26945068+25.4
AURORA Multilingual Terminal-BenchCode49.173.7+24.6
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine16.040.0+24.0
EnigmaEvalGeneral QA20.543.9+23.4
Mystery Game PuzzlesGames36.059.0+23.0
GameBench 2Games62.584.8+22.4
Humanity's Last Exam DiamondGeneral QA23.144.8+21.7
HiL-Bench (Human-in-Loop Benchmark)Agents35.357.0+21.7
BullshitBench v2 LeaderboardSafety94.073.0−21.0
SWE-bench MultimodalCode38.459.4+21.0
RiemannBenchMath47.268.0+20.8
OSWorld 2.0 - reported partial reward, patch and step limit unspecifiedAgents54.875.4+20.6
KernelBench CUDA Normalized (RTX PRO 6000)Code24.945.5+20.5
EBR-bench v2Games25.245.7+20.5
SimpleQA VerifiedGeneral QA39.559.9+20.4
Harvey's Legal Agent Benchmark - Capital MarketsLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal40.020.0−20.0
APEX-SWECode43.963.7+19.8
TasteVal (compute multiplier)Science0.81.3+19.5
MindCube TinyMultimodal64.984.3+19.4
AA-Briefcase EloAgents13161645+19.1
NanoGPT Speedrun Frontier (human record gap closed)Agents39.453.6+19.1
OSWorld 2.0 - reported binary completion, patch and step limit unspecifiedAgents20.639.6+19.0
DeepSWE v1.1Code54.473.2+18.8
SWE-bench Verified Extended (Mercor held-out)Code63.382.0+18.7
LisanBench Path LengthGames946315428+18.5
ALE-BenchCode21022622+18.4
ARC-AGI-2General QA72.190.4+18.3
MineBench Conservative RatingGames16861995+18.3
Furniture Assembly BenchmarkMultimodal42.560.8+18.3
ZeroBench (Main questions, pass@5, with tools)Multimodal34.052.0+18.0
OSS-Fuzz (Anthropic)Cybersecurity61.579.4+17.9
AutoformBenchMath44.061.9+17.9
LMArena Code Arena WebDevCode15551691+17.7
JobBench MainProfessional Work48.465.7+17.3
PCT Political Inconsistency (CAIS)Safety53.836.7+17.1
FrontierMath Tier 4 (v2) - PrivateMath56.173.2+17.1
EnterpriseBench: CoreCraftProfessional Work52.368.7+16.4
GDPval-AA v2Professional Work15291861+16.3
MRCR v2 (8-needle, Context Arena) - 128K pointwiseGeneral QA75.291.3+16.2
BlenderWorldCode57.073.0+16.0
SWE Atlas Codebase QnA Extended (Mercor held-out)Code24.040.0+16.0
Blueprint-Bench 2Multimodal0.10.3+15.9
ChessBench CoherenceGames79.763.8−15.9
SimpleBenchGeneral QA64.880.6+15.8
OSWorld 2.0 500-step Partial ScoreAgents54.870.6+15.8
HPCT (SecureBio)Science63.178.6+15.5
HPCT Refusal (CAIS)Safety85.0100.0+15.0
SUPERChemScience60.374.7+14.4
ScBench-LongScience25.439.7+14.3
GBENCH One-shot Coding GScoreGames54.068.1+14.1
BrowseComp Extended (Mercor held-out)Search30.044.0+14.0
Medical Long Context Reasoning (MLCR-AA)Health & Medicine45.659.4+13.9
MineBench — Bradley-Terry RatingGames17862008+13.4
GDPval-AA v2 (normalized)Professional Work54.567.9+13.4
FrontierCode 2.0 - Pass RateCode17.130.4+13.3
LLM Debate BenchmarkChat & Writing16311724+13.3
TextQuests Game ProgressGames40.153.3+13.2
HealthBench ProfessionalHealth & Medicine60.373.4+13.1
BabyVision with toolsMultimodal81.294.1+12.9
SM BenchSafety72.985.8+12.9
Terminal-Bench Science 0.1 (Vals)Science10.022.9+12.9
Revelo Code IndexCode35.648.3+12.7
Roboflow Vision Evals (October 2026, five tasks)Multimodal58.370.9+12.6
APEX-Agents 1.1Professional Work64.477.0+12.6
SWE Atlas - Test WritingCode49.662.2+12.6
ReactBench v1Code36.549.0+12.5
MedXpertQA MM Extended (Mercor held-out)Health & Medicine51.664.1+12.5
CadQueryEval (Corrected Geometry)Code68.080.0+12.0
BankerToolBench v1.1Finance & Accounting20.732.6+11.9
FrontierCode 2.0 - ScoreCode15.327.2+11.9
GDPval-AA v2.1Professional Work14611729+11.8
VCT-Refusal (CAIS)Safety41.229.4−11.8
Tax Agent BenchFinance & Accounting64.976.7+11.7
TxBench Oligo Discovery V1 Clean RerunsScience33.945.6+11.7
HLE Extended (Mercor held-out)General QA49.461.0+11.6
Legal Research BenchLegal43.855.3+11.5
DeepResearch Bench II (Mercor 130-task set)Search44.656.1+11.5
Design Arena Overall Code CategoriesCode12521321+11.4
Bug Hunt BenchCode14.325.7+11.4
ERQAMultimodal59.570.8+11.3
AutomationBench-AA 1.0.6Agents45.656.6+11.0
BuseyBench SVGMultimodal5.26.3+10.9
AA-LCR Extended (Mercor held-out)Search63.273.9+10.7
ScBench-Long V0 VerifiedScience25.836.4+10.6
Bio Capabilities Index (SecureBio)Indexes164178+10.6
SpatialViz-BenchMultimodal65.275.8+10.6
𝜏³-BankingAgents34.244.7+10.5
VariantBench V1 Clean RerunsScience42.452.8+10.5
HANDBOOK.mdProfessional Work21.932.3+10.4
MedCodeHealth & Medicine53.263.6+10.4
Ramp SWE-Bench (78 published tasks)Code78.288.5+10.3
Code MigrationCode47.357.5+10.2
AutomationBench 1.0.6 (Zapier)Agents16.926.9+10.1
IntPhys 2Multimodal55.165.1+10.0
HardGeoBench <=100miMultimodal36.346.3+10.0
SWE-Bench Pro (Public Dataset)Code69.279.2+10.0
Agents' Last Exam (ALE)Agents22.432.2+9.9
MRCR v2 (8-needle, Context Arena) - 128K AUCGeneral QA87.897.5+9.7
Roboflow Vision Evals (September 2026 protocol)Multimodal68.778.3+9.6
RuneBenchGames5.15.7+9.6
Vals RSI Index v1.1 (four tasks)Agents24.033.0+9.0
Chess Puzzles (task version 2.0)Games52.043.0−9.0
ProofBenchMath69.078.0+9.0
ZeroBench (Main questions, pass@5)Multimodal17.026.0+9.0
Artificial Analysis Intelligence Index v4.3.2Indexes41.850.8+9.0
CorpFin v2 - Shared Max ContextFinance & Accounting66.074.9+9.0
CorpFin v2 (Vals Index component)Finance & Accounting66.074.9+9.0
CorpFin v2 (Vals Multimodal Index component)Finance & Accounting66.074.9+9.0
Antidote: Everyday EditionSafety10301055+8.9
EurekaBenchScience18.427.3+8.9
WeirdML (v2)Science82.991.8+8.9
CyberArenaCybersecurity65.073.8+8.9
Vals RSI Index v1.1 (historical five tasks)Agents23.232.1+8.9
LMArena Agent Arena Net ImprovementAgents6.99.5+8.8
Public Benefits Bench v1.1Professional Work68.176.9+8.8
AA-AnalystAgent (pass^5)Professional Work45.053.8+8.8
Conceptual Reasoning IndexIndexes68.374.7+8.7
Artificial Analysis Intelligence Index v4.3Indexes42.050.7+8.7
ChessBench — Historical Output RepairGames24.032.7+8.7
DrugDiscoveryBenchScience45.153.7+8.6
SWE-bench VerifiedCode88.697.0+8.4
Harvey LAB (Mercor run, 1,749 public tasks)Legal9.417.8+8.4
HealthBenchHealth & Medicine58.867.1+8.3
TxBench Antibody Discovery V1 Clean RerunsScience32.440.6+8.3
ActiveVisionMultimodal2.410.6+8.2
TxBench Oligo Discovery V0Science39.948.1+8.2
ExploitGymCybersecurity13.822.0+8.2
LVBenchMultimodal67.375.4+8.1
Artificial Analysis Agentic IndexIndexes47.255.3+8.1
ChessBench EloGames961826−8.0
Chartography (with tools)Multimodal75.083.0+8.0
Co-Design BenchCode58.166.1+8.0
ATLAS-Education (StudentBench) Lesson PlanningProfessional Work0.91.1+8.0
AutomationBench (Overall)Agents17.225.1+7.9
WitnessBench v1.0Games22.830.6+7.8
CursorBench 3.2Code62.370.0+7.7
VariantBenchScience42.149.4+7.3
MedXpertQA MM (200-question set)Health & Medicine72.579.8+7.3
NoCoT-Bench NCRI 15.2General QA102115+7.2
Ruby LLM Benchmarks — Program FixerCode72.679.8+7.2
SCI-AGENT-EXAMScience57.164.2+7.1
Terminal-Bench 2.1Code82.089.1+7.1
Humanity's Sixth SenseMultimodal23.530.5+7.0
Time Horizon Index: KSPAgents11.818.8+7.0
YC-BenchGames17113891911592+7.0
LMCAGeneral QA57.564.5+6.9
AdvancedIFChat & Writing73.780.5+6.8
Into the Unknown Direction Node Pass RateScience60.166.8+6.7
Humanity's Last ExamGeneral QA49.856.4+6.6
APEX-AccountingFinance & Accounting48.054.5+6.5
BrowseCompSearch84.390.8+6.5
ACCoRDGeneral QA77.483.8+6.5
CorpFin v2 - Max Fitting ContextFinance & Accounting69.876.0+6.2
ABC-Bench liquid handling (SecureBio)Science100.094.0−6.0
SWE Atlas - Codebase QnACode57.363.2+5.9
SpatialBench VerifiedScience66.672.5+5.9
APEX-1 (Mercor, mean score)Professional Work62.768.6+5.9
Wiz Cyber Model Arena v3.9Cybersecurity65.471.2+5.8
Into the Unknown Integrity Node Pass RateScience65.359.5−5.8
TxBench Antibody Discovery V0Science45.651.4+5.8
Vibe Code Bench v1.1Code82.788.4+5.7
FrontierMath Tiers 1-3 (v2) - PrivateMath80.085.6+5.6
NL2Repo - release-reported repository generation scoreCode69.775.3+5.6
TextQuests Average Cumulative HarmSafety21.022.6−5.6
Artificial Analysis Progress Index v1Indexes318334+5.4
SAGEMath54.849.4−5.4
SpatialBench-Long V1 Clean RerunsScience33.328.0−5.3
TxBench Preclinical Pharmacology V1 Clean RerunsScience57.763.0+5.3
LiveBench 2.0Agents76.882.1+5.3
Terminal-Bench 2.1 Extended (Mercor held-out)Code32.738.0+5.3
BioTIER RefusalSafety94.7100.0+5.3
MedScribeHealth & Medicine85.891.0+5.2
CharXiv ReasoningMultimodal78.583.7+5.2
A-Fantasia (Valid Attempts with Retries)Games34.729.7+5.0
InteractiveBench (task pass)Agents28.333.3+5.0
ARC-AGI-1 Semi-PrivateGeneral QA92.597.5+5.0
HardGeoBench <=10miMultimodal27.532.5+5.0
Finance Agent v2Finance & Accounting53.958.6+4.7
Drone-BenchAgents82.386.9+4.6
BioSecBench-Surveillance V1 Clean RerunsScience46.951.5+4.6
PWNBench v0.1Cybersecurity55.660.0+4.4
CorpFin v2 - Exact PagesFinance & Accounting64.368.6+4.3
RedlineBenchLegal39.744.0+4.3
Senior SWE-BenchCode30.534.7+4.2
LAB-Bench ProtocolQA (SecureBio)Science76.981.1+4.2
Excel Modeling BenchmarkFinance & Accounting69.473.6+4.2
AA-Omniscience IndexIndexes28.837.1+4.2
GlobeBenchGeneral QA20.015.9−4.1
LAB-Bench CloningScenarios (SecureBio)Science95.9100.0+4.1
BabaIsBench Stage 0: The IntroGames96.0100.0+4.0
GDPval Extended (Mercor held-out rubric score)Professional Work58.962.9+4.0
GeneBench-Pro Public Release SubsetScience18.022.0+4.0
MBCT (SecureBio)Science56.460.2+3.8
EpiBenchScience39.035.2−3.8
Artificial Analysis Coding IndexIndexes74.378.0+3.7
Agent Red Teaming k=100 (CAIS)Safety47.043.3+3.7
Epoch Capabilities Index (ECI)Indexes160164+3.7
SpeciEval (Pooled English Runs)Safety84.387.9+3.6
SpeciEval (Refusal-Aware Retries, Historical)Safety84.387.9+3.6
MCP Atlas April 2026 (all 1,000 tasks)Agents82.285.8+3.6
EnterpriseOps-Gym-AAAgents44.047.5+3.5
ScBench-Long V1 Clean RerunsScience21.024.4+3.4
LegalBenchLegal83.687.0+3.4
MetagenomicsBench V1Science50.053.3+3.3
MetagenomicsBench V0Science50.053.3+3.3
MMMU-Pro (Vals 4-Option)Multimodal86.689.9+3.3
Extended NYT Connections 940 (Quadratic v1)Games91.194.3+3.2
VCT (SecureBio)Science48.251.4+3.2
LMArena Agent Arena Bash RecoveryAgents7.213.6+3.2
CharXiv Extended (Mercor held-out)Multimodal76.779.8+3.1
SpatialBench-Long V0 VerifiedScience28.831.8+3.0
ProgramBench (Anthropic 166-task, 5-episode hidden-test pass rate)Code90.093.0+3.0
DTBenchGeneral QA94.997.9+2.9
MMMU-Pro Extended (Mercor held-out)Multimodal31.434.3+2.9
LAB-Bench LitQA2 (SecureBio)Science48.751.4+2.7
PG-LLM-50Science0.40.4+2.5
Hemingway-benchChat & Writing10381047+2.5
DeepSearchQA (F1 score)Search92.294.6+2.4
scBenchScience58.260.6+2.4
SWE-TogetherCode61.563.8+2.3
MortgageTax (Vals all 1,258 documents)Finance & Accounting69.972.1+2.1
WCB (SecureBio)Science66.768.8+2.1
SpatialBench V0 VerifiedScience69.071.0+2.0
MMLU ProGeneral QA90.092.0+2.0
WordleBenchGames94.096.0+2.0
SciCode 1.0.1 (Artificial Analysis)Code54.456.4+2.0
GPQA-Bio (SecureBio)Science80.982.7+1.8
ATLAS-Education (StudentBench) Learning GainProfessional Work16.012.6−1.7
AIME 2025Math98.3100.0+1.7
ComplexConstraintsChat & Writing35.637.3+1.7
BioSecBench-Function V0Science44.643.0−1.6
NoCoT-Bench NCKI (kspine v3)General QA117119+1.5
ChessBench AccuracyGames72.471.0−1.5
SpatialBench V1 Clean RerunsScience67.068.4+1.5
Tampered MLScience76.577.9+1.4
CharXiv (descriptive and reasoning combined)Multimodal93.294.6+1.4
Toloka Arena (v1)Agents54.352.9−1.4
LMArena Text Arena EloChat & Writing14821490+1.4
SWE-Marathon v1.1Code48.850.0+1.3
Bench to the Future 3 — Pooled ScoreForecasting0.10.1+1.2
DistressBenchSafety71.470.2−1.2
ScBench V1 Clean RerunsScience57.158.3+1.2
Bench to the Future 3 — September Panel — PooledForecasting0.10.1+1.1
GDPval (Mercor 213-task rubric score)Professional Work84.585.6+1.1
Arena Alignment IndexSafety74.575.6+1.1
MASKSafety94.493.4−1.0
HardGeoBench Avg DistanceMultimodal12451218+0.8
BioSecBench-RefusalSafety28.629.4+0.8
LiveCodeBench (Vals Public) - MediumCode89.890.6+0.8
ObviousBenchGeneral QA99.3100.0+0.7
EpiBench V1 Clean RerunsScience25.825.2−0.6
BioSecBench-Function V1 Clean RerunsScience43.142.5−0.6
HieroglyphBenchMultimodal14.014.6+0.6
OTIS Mock AIME 2024-2025Math98.398.9+0.6
LMArena Agent Arena SteerabilityAgents9.610.7+0.5
FrontierCode 1.1 Extended - Pass RateCode65.565.0−0.5
GPQA DiamondGeneral QA91.992.4+0.5
TaxEval v2Finance & Accounting75.675.1−0.5
SciCode (main problems)Science16.917.3+0.4
ForecastBench Preliminary Dataset Brier IndexForecasting61.662.0+0.4
ForecastBench Tournament Overall Brier IndexForecasting66.266.6+0.4
MMLU-Bio (SecureBio)Science90.991.1+0.2
Harvey HLAB — Criteria Pass RateLegal87.987.7−0.2
Vision Arena OverallMultimodal12861285−0.1
OSWorld-VerifiedAgents83.483.40.0
GDP.pdfProfessional Work24.024.00.0
HardGeoBench Country AccuracyMultimodal53.853.80.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal20.020.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.00.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal20.020.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
Into the UnknownScience0.00.00.0
LiveCodeBench (Vals Public) - EasyCode99.199.10.0
Olam Social Poker EloGames151815180.0
TaxCalcBench TY25Finance & Accounting20.020.00.0

Opus 4.8vsClaude Opus 5normalised score marginsData Nov 15, 2026Data: Vals AI, Mercor, benchmarks.bio and 88 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…