Atlas

Compare Models

Claude Fable 5.1 vs Claude Opus 5

Claude Fable 5.1 leads by 1.7 capability points (≈95% interval −0.8 to +4.1): statistically tied · P(higher) ≈ 91%

Wins 173 of 242 shared benchmarks (60 lost, 9 tied)

Capability vs. relative cost
Claude Fable 5.1Claude Opus 5Closed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Fable 5.1vsClaude Opus 5capability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins173 – 60 · 242 shared, 9 tied

Comparing matching setups only (7 benchmarks excluded) · suites counted once
← Claude Fable 5.1Claude Opus 5 →

Claude Fable 5.1vsClaude Opus 5wins by categoryData Nov 15, 2026Data: Mercor, Vals AI, Arena and 78 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Claude Opus 5 minus Claude Fable 5.1.

VCT-Refusal (CAIS)Safety98.929.4−69.5
Time Horizon Index: KSPAgents63.318.8−44.5
Machiavelli Deception (CAIS)Safety82.496.5−42.5
SPU (Science Prediction Universe)Science-0.50.2+35.3
BioSecBench-Refusal V1 Clean RerunsSafety0.031.8+31.8
Anthropic ECI (AECI), November 2026 fitIndexes173169−28.2
Olam Social Poker EloGames14941518+26.1
Terminal-Bench-Science 0.1Science52.630.0−22.6
TaxCalcBench TY25Finance & Accounting42.020.0−22.0
SWE-bench Verified Extended (Mercor held-out)Code61.382.0+20.7
TasteVal (compute multiplier)Science1.71.3−18.9
Terminal-Bench 2.1 Extended (Mercor held-out)Code19.538.0+18.5
FoodTruck BenchGames6273275264+16.1
ChessBench EloGames1095826−16.0
LABBench2Science68.684.2+15.6
Bug Hunt BenchCode41.025.7−15.2
TextQuests Average Cumulative HarmSafety27.122.6+15.2
LMArena Agent Arena Net ImprovementAgents14.19.5−14.9
BioSecBench-Structure V1 (58 evaluations)Science5.820.7+14.9
SciUniverse Level 1Science45.330.5−14.8
FrontierMath Tier 4 (v2) - PrivateMath87.873.2−14.6
LMArena Agent Arena Net Improvement (Steering Burden)Agents14.87.2−14.3
MMMU-Pro Extended (Mercor held-out)Multimodal47.634.3−13.3
Hemingway-benchChat & Writing10951047−13.3
Arena Image-to-WebDevCode17241664−13.0
Humanity's Last Exam DiamondGeneral QA51.338.6−12.7
HardGeoBench <=10miMultimodal45.032.5−12.5
Medical Long Context Reasoning (MLCR-AA)Health & Medicine71.159.4−11.7
HardGeoBench Avg DistanceMultimodal8451218−11.5
Blueprint-Bench 2Multimodal0.40.3−11.5
Terminal-Bench Science 0.1 (Vals)Science34.322.9−11.4
EBR-bench v4 (card ban, single agent)Games57.145.7−11.4
InferenceBenchCode9.88.9−11.1
BioSecBench-RespiratoryVirus-Surveillance V1 (110 evaluations)Science3.614.6+10.9
MysteryMechanismScience47.737.4−10.4
MedCodeHealth & Medicine53.563.6+10.1
τ³-Banking (Mercor run, bm25_grep retrieval)Agents54.044.0−10.0
FrontierSWE V2 (Mercor rerun, mean@3)Code37.227.3−9.9
Humanity's Last ExamGeneral QA46.556.4+9.9
LMArena Agent Arena Praise vs ComplaintAgents31.612.2−9.7
SnorkelLegalLegal35.926.4−9.5
GameBench 2Games94.084.8−9.2
Furniture Assembly BenchmarkMultimodal70.060.8−9.2
Agent Red Teaming k=100 (CAIS)Safety34.243.3−9.1
Concept Synth INDUCTION Challenge100 Round 1General QA33.024.0−9.0
Toloka Arena (v3)Agents71.062.0−9.0
EnterpriseBench: CoreCraftProfessional Work77.468.7−8.7
DAYJOB: FinanceFinance & Accounting19.811.3−8.5
InteractiveBench (task pass)Agents41.733.3−8.3
ALE-BenchCode28492622−8.0
Eval Connections One-Shot with TrapsGames82.074.0−8.0
ComplexConstraintsChat & Writing45.137.3−7.8
Toloka Arena (v1)Agents60.752.9−7.8
HardGeoBench <=100miMultimodal53.846.3−7.5
MASKSafety85.993.4+7.5
Terminal-Bench 2.1Code81.689.1+7.5
BuseyBench SVGMultimodal7.06.3−7.3
ReactBench (39 tasks)Code49.242.1−7.2
FrontierCode 2.0 - Pass RateCode37.630.4−7.1
NoCoT-Bench NCRI 15.2General QA128115−7.1
ProgramBench (Mercor 100-task subset)Code11.34.3−7.0
MA-BenchProfessional Work49.256.2+7.0
Shroombench (mushroom identification, top-1)Multimodal51.044.0−7.0
WeirdML v3Science26.019.0−7.0
SciCode (main problems)Science24.217.3−6.9
RuneBenchGames6.15.7−6.7
SciCode 1.0.1 (Artificial Analysis)Code63.156.4−6.7
OSWorld 2.1 full - reported partial reward, step limit unspecifiedAgents80.774.0−6.7
LMArena Code Arena WebDevCode17421691−6.7
BabaIsBench Stage 1: The LakeGames100.093.3−6.7
FrontierCode 2.0 - ScoreCode34.027.4−6.7
HANDBOOK.mdProfessional Work38.832.3−6.5
APEX-AccountingFinance & Accounting61.054.5−6.5
BrowseComp (Mercor 130-question subset)Search78.384.6+6.3
NoCoT-Bench NCKI (kspine v3)General QA128119−6.2
KernelBench CUDA Normalized (RTX PRO 6000)Code51.645.5−6.2
ClockBench AccuracyMultimodal59.465.5+6.1
SimpleBenchGeneral QA86.680.6−6.0
LiveBench 2.0Agents88.082.1−5.9
GDPval Extended (Mercor held-out rubric score)Professional Work68.762.9−5.8
HieroglyphBenchMultimodal20.414.6−5.8
SnorkelUnderwrite 2.0Finance & Accounting24.018.3−5.7
GlobeBenchGeneral QA21.615.9−5.7
Document ArenaProfessional Work15151509−5.5
AA-LCR v1.1Search87.582.0−5.5
SWE-TogetherCode69.363.8−5.5
PostTrainBench Average (v1.1)Indexes40.245.6+5.4
DAXBench v2Code91.586.2−5.3
LisanBench Difficulty-WeightedGames45735068+5.3
LMArena Agent Arena Steering BurdenAgents14.84.4−5.2
APEX-1 (Mercor, mean score)Professional Work73.868.6−5.2
CursorBench 4.0Code51.846.6−5.2
Chess Puzzles (task version 2.0)Games48.043.0−5.0
HardGeoBench Country AccuracyMultimodal58.853.8−5.0
HackerRank-BenchCode47.542.6−4.9
MineBench — Bradley-Terry RatingGames19282008+4.8
SWE Atlas - Test WritingCode67.062.2−4.8
ActiveVisionMultimodal15.310.6−4.7
AutomationBench 1.0.6 (Zapier)Agents22.426.9+4.6
FrontierMath Tiers 1-3 (v2) - PrivateMath90.285.6−4.6
Arena Alignment IndexSafety80.175.6−4.6
HiL-Bench (Human-in-Loop Benchmark)Agents61.557.0−4.5
AutomationBench - release-reported metric and version unspecifiedAgents31.426.9−4.5
AutomationBench (Overall)Agents31.426.9−4.5
Extended NYT Connections 940 (Quadratic v1)Games90.094.3+4.3
LVBenchMultimodal79.775.4−4.3
Drone-BenchAgents91.286.9−4.3
HealthBench ProfessionalHealth & Medicine69.273.4+4.2
BrokenArXiv 06/2026Math86.690.7+4.2
CUA-benchAgents13.29.0−4.2
BullshitBench v2 LeaderboardSafety77.073.0−4.0
CadQueryEval (Corrected Geometry)Code76.080.0+4.0
GeneBench-Pro Public Release SubsetScience26.022.0−4.0
Terminal-Bench 4.0Code57.953.9−3.9
MedXpertQA MM (200-question set)Health & Medicine83.779.8−3.9
MBABench 2.0Finance & Accounting7.94.0−3.9
Ruby LLM Benchmarks — Program FixerCode76.079.8+3.9
Roboflow Vision Evals (October 2026, five tasks)Multimodal74.770.9−3.8
AA-AnalystAgent (pass^5)Professional Work57.553.8−3.7
AdvancedIFChat & Writing84.280.5−3.7
Clipping Bench v1.0Games1068734−3.6
Antidote: Everyday EditionSafety10651055−3.6
CursorBench 3.2Code73.470.0−3.4
PCT Political Inconsistency (CAIS)Safety33.436.7−3.3
TextQuests Game ProgressGames56.653.3−3.3
SM BenchSafety89.085.8−3.2
HealthBenchHealth & Medicine63.967.1+3.2
SWE Atlas - Codebase QnACode60.063.2+3.2
SWE-Marathon v1.1Code45.648.8+3.1
FORTRESS Average Risk Score (ARS)Safety13.716.8−3.1
Excel Modeling BenchmarkFinance & Accounting76.773.6−3.1
micro1 PharmaBenchHealth & Medicine12.89.7−3.1
SnorkelFinance 2.0Finance & Accounting19.616.5−3.1
Vals RSI Index v1.1 (four tasks)Agents36.133.0−3.1
Roboflow Vision Evals (September 2026 protocol)Multimodal81.378.3−3.0
SWE-Clar (implementation)Code60.063.0+3.0
Vals RSI Index v1.1 (historical five tasks)Agents35.032.1−2.9
BenchCAD - geometric overlapMultimodal92.689.9−2.7
Artificial Analysis Intelligence IndexIndexes65.763.1−2.6
IntPhys 2Multimodal67.765.1−2.6
OSWorld 2.0 - reported partial reward, patch and step limit unspecifiedAgents77.975.4−2.5
EurekaBenchScience29.827.3−2.5
Artificial Analysis Coding Agent Index v1.5Indexes62.259.7−2.5
CharXiv Extended (Mercor held-out)Multimodal82.279.8−2.4
RiemannBenchMath65.668.0+2.4
AutoResearchExam (hidden-test AUARC)Agents0.60.6−2.3
Biopharma Bench V0.1Health & Medicine69.967.6−2.3
DeepSWE v1.1Code67.365.0−2.3
LMArena Text Arena EloChat & Writing15031490−2.3
SWE-Shift (Shifted)Code18.316.1−2.2
Hedge-BenchFinance & Accounting26.829.0+2.2
SpatialViz-BenchMultimodal77.975.8−2.1
LMArena Agent Arena Confirmed SuccessAgents14.410.2−2.1
MMMU ProMultimodal86.184.0−2.1
OSWorld 2.0 - reported binary completion, patch and step limit unspecifiedAgents41.739.6−2.1
Epoch Capabilities Index (ECI)Indexes167164−2.1
CritPt (avg@5)Science31.129.1−2.0
OpenAI internal computer-use safety - adverse outcome rateSafety9.511.5−2.0
SWE Atlas Codebase QnA Extended (Mercor held-out)Code38.040.0+2.0
SAB RankingForecasting-0.1-0.1−2.0
LLM Debate BenchmarkChat & Writing17381724−1.9
ChessBench CoherenceGames61.963.8+1.9
MedXpertQA MM Extended (Mercor held-out)Health & Medicine66.064.1−1.9
SnorkelManufacturingProfessional Work11.39.4−1.9
Snorkel Digital Work IndexIndexes23.621.8−1.9
GDPval-AA v2Professional Work17451708−1.8
SnorkelRevOpsProfessional Work14.012.2−1.8
FrontierPhysics v0.1Science11.39.5−1.8
RSI-Exam 0.1 (Full 88)Agents0.50.5−1.7
Design Arena Overall Code CategoriesCode13311321−1.7
GDPval-AA v2.1Professional Work17651729−1.6
Concept Synth INDUCTION Challenge64 Round 1General QA34.432.8−1.6
LegalBenchLegal88.587.0−1.5
LiveCodeBench (Vals AI public dataset)Code90.589.0−1.5
CritPt v2 (agentic, avg@5)Science72.971.4−1.4
Search ArenaSearch12371239+1.4
DistressBenchSafety71.570.2−1.3
MortgageTax (Vals all 1,258 documents)Finance & Accounting70.872.1+1.3
Tax Agent BenchFinance & Accounting77.976.7−1.2
DAYJOB: HealthcareHealth & Medicine9.68.4−1.2
GBENCH One-shot Coding GScoreGames69.368.1−1.2
LMArena Agent Arena SteerabilityAgents8.410.7+1.1
WeirdML (v2)Science92.991.8−1.1
OTIS Mock AIME 2024-2025Math100.098.9−1.1
DeepResearch Bench II (Mercor 130-task set)Search55.056.1+1.1
Creative Writing v3 EloChat & Writing21552135−1.0
LisanBench Path LengthGames1510015428+1.0
LMCAGeneral QA65.564.5−1.0
Mystery Game PuzzlesGames58.059.0+1.0
GDPval (Mercor 213-task rubric score)Professional Work86.685.6−1.0
Longform Creative WritingChat & Writing85.386.3+1.0
MMLU ProGeneral QA93.092.0−1.0
ProofBench v1.1Math100.099.0−1.0
Vision Arena OverallMultimodal12891285−1.0
AA-Briefcase EloAgents16621645−1.0
LMArena Agent Arena Bash RecoveryAgents11.613.6+1.0
AA-LCR Extended (Mercor held-out)Search73.073.9+0.9
Snorkel WorkplaceAgentsProfessional Work15.814.9−0.9
SAGEMath48.549.4+0.9
FrontierCode 1.1 Extended - Pass RateCode68.869.6+0.8
TaxEval v2Finance & Accounting76.075.1−0.8
Snorkel Agentic Coding 2.0Code39.638.9−0.7
SpeciEval (Refusal-Aware Retries, Historical)Safety87.287.9+0.7
SpeciEval (Pooled English Runs)Safety87.287.9+0.7
ObviousBenchGeneral QA99.3100.0+0.7
SAB ResearchFinance & Accounting96.896.2−0.6
ERQAMultimodal70.370.8+0.5
GPQA DiamondGeneral QA92.992.4−0.5
BioTIER RefusalSafety99.5100.0+0.5
CharXiv (descriptive and reasoning combined)Multimodal95.194.6−0.5
Snorkel SWE-bench CLICode14.514.0−0.5
ARC-AGI-2General QA90.090.4+0.4
CheatBenchSafety45.845.3+0.4
Conceptual Reasoning IndexIndexes74.974.7−0.4
CVDP (517-problem set)Code62.662.2−0.4
Realm Financial ReasoningFinance & Accounting47.847.4−0.4
ATLAS-Finance v1.1Finance & Accounting10.310.7+0.3
ATLAS-Finance v1.0Finance & Accounting12.012.3+0.3
ChessBench AccuracyGames71.371.0−0.3
MedScribeHealth & Medicine91.391.0−0.3
SWE-Bench Pro V2 — FullCode99.199.4+0.3
EnigmaEvalGeneral QA44.243.9−0.3
SWE-Clar (clarification)Code84.384.0−0.3
LLM Creative Story WritingChat & Writing3.83.7−0.3
DTBenchGeneral QA97.697.9+0.3
Harvey LAB-AA v1.0 (criterion pass rate)Legal93.393.5+0.2
MindCube TinyMultimodal84.584.3−0.2
ACCoRDGeneral QA83.783.8+0.2
LMArena Agent Arena Bash ReliabilityAgents10.19.8−0.1
APEX-SWECode63.663.7+0.1
APEX-Agents 1.1Professional Work77.177.0−0.1
LMArena Agent Arena Tool HallucinationAgents0.80.9+0.1
Creative Writing v3 Rubric ScoreChat & Writing16.316.30.0
AIME 2025Math100.0100.00.0
ARC-AGI-1 Semi-PrivateGeneral QA97.597.50.0
BabaIsBench Stage 0: The IntroGames100.0100.00.0
FrontierCode 1.1 Extended - ScoreCode63.663.60.0
FrontierMath ErdősMath0.00.00.0
HPCT Refusal (CAIS)Safety100.0100.00.0
Senior SWE-BenchCode34.734.70.0
SUPERChemScience74.774.70.0
SWEeper-BenchCode56.556.50.0

Claude Fable 5.1vsClaude Opus 5normalised score marginsData Nov 15, 2026Data: Mercor, Vals AI, Arena and 78 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…