Atlas

Compare Models

Claude Fable 5 vs Claude Fable 5.1

Claude Fable 5.1 leads by 2.3 capability points (≈95% interval −0.2 to +4.7): statistically tied · P(higher) ≈ 96%

Wins 115 of 168 shared benchmarks (44 lost, 9 tied)

Capability vs. relative cost
Claude Fable 5Claude Fable 5.1Closed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Fable 5vsClaude Fable 5.1capability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins44 – 115 · 168 shared, 9 tied

Comparing matching setups only (53 benchmarks excluded) · suites counted once
← Claude Fable 5Claude Fable 5.1 →

Claude Fable 5vsClaude Fable 5.1wins by categoryData Nov 15, 2026Data: Mercor, Arena, Center for AI Safety and 53 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Claude Fable 5.1 minus Claude Fable 5.

Olam Social Poker EloGames15531494−64.1
BenchCAD - geometric overlapMultimodal38.492.6+54.2
BrokenArXiv 06/2026Math47.886.6+38.7
SWE Atlas Codebase QnA Extended (Mercor held-out)Code0.038.0+38.0
Anthropic ECI (AECI), November 2026 fitIndexes169173+34.5
Furniture Assembly BenchmarkMultimodal35.870.0+34.2
BrokenArXiv 05/2026Math44.578.0+33.5
Terminal-Bench-Science 0.1Science21.452.6+31.2
Arena Image-to-WebDevCode16211724+22.5
ClockBench AccuracyMultimodal37.259.4+22.2
Terminal-Bench Science 0.1 (Vals)Science12.934.3+21.4
BullshitBench v2 LeaderboardSafety56.077.0+21.0
SWE Atlas - Codebase QnACode39.060.0+21.0
ChessBench CoherenceGames82.661.9−20.7
LMArena Agent Arena Net ImprovementAgents8.014.1+20.0
AdvancedIFChat & Writing65.284.2+19.0
GameBench 2Games75.294.0+18.8
SWE-bench Verified Extended (Mercor held-out)Code78.761.3−17.4
Document ArenaProfessional Work14971515+16.3
LMArena Code Arena WebDevCode16251742+15.3
GlobeBenchGeneral QA7.321.6+14.3
LMArena Agent Arena Net Improvement (Steering Burden)Agents7.314.8+14.2
ALE-BenchCode24632849+13.7
MMLU ProGeneral QA79.493.0+13.6
MMMU ProMultimodal72.786.1+13.4
Bug Hunt BenchCode27.641.0+13.3
Terminal-Bench 4.0Code44.557.9+13.3
Antidote: Everyday EditionSafety10991065−12.1
TaxCalcBench TY25Finance & Accounting30.042.0+12.0
GeneBench-Pro Public Release SubsetScience14.026.0+12.0
Vals RSI Index v1.1 (four tasks)Agents24.336.1+11.8
TextQuests Average Cumulative HarmSafety23.727.1−11.5
SWE Atlas - Test WritingCode55.667.0+11.4
ChartographyMultimodal34.846.2+11.4
ProgramBench (Mercor 100-task subset)Code0.011.3+11.3
Humanity's Last ExamGeneral QA57.846.5−11.3
DAYJOB: FinanceFinance & Accounting9.019.8+10.8
Vals RSI Index v1.1 (historical five tasks)Agents24.535.0+10.6
WeirdML v3Science15.626.0+10.4
SUPERChemScience64.474.7+10.3
LLM Creative Story WritingChat & Writing2.93.8+10.2
MMMU-Pro Extended (Mercor held-out)Multimodal37.647.6+10.0
GSO Opt@1Code78.488.2+9.8
MirrorCode (ML, +Private, 2L)Code63.973.3+9.4
MedXpertQA MM Extended (Mercor held-out)Health & Medicine56.966.0+9.1
AA-AnalystAgent (pass^5)Professional Work48.857.5+8.7
LMArena Agent Arena Praise vs ComplaintAgents14.331.6+8.7
ActiveVisionMultimodal7.115.3+8.2
Tax Agent BenchFinance & Accounting69.877.9+8.1
Concept Synth INDUCTION Challenge64 Round 1General QA42.234.4−7.8
APEX-1 (Mercor, mean score)Professional Work66.073.8+7.8
AA-Briefcase EloAgents15301662+7.7
MedXpertQA MM (200-question set)Health & Medicine76.283.7+7.5
Search ArenaSearch12281237+7.3
RuneBenchGames5.76.1+7.1
EnterpriseBench: CoreCraftProfessional Work70.377.4+7.1
ComplexConstraintsChat & Writing38.145.1+7.0
MASKSafety92.885.9−6.9
Drone-BenchAgents84.391.2+6.8
InferenceBenchCode8.79.3+6.8
Machiavelli Deception (CAIS)Safety80.282.4−6.7
FrontierFinance v3.1Finance & Accounting49.255.9+6.7
Medical Long Context Reasoning (MLCR-AA)Health & Medicine64.471.1+6.7
GDPval-AA v2Professional Work16311764+6.5
Terminal-Bench 2.1Code88.081.6−6.4
PCT Political Inconsistency (CAIS)Safety39.833.4+6.4
ChessBench AccuracyGames77.171.3−6.2
CharXiv (descriptive and reasoning combined)Multimodal89.195.1+6.0
Mystery Game PuzzlesGames52.058.0+6.0
RiemannBenchMath60.065.6+5.6
ArXivMath 06/2026Math85.491.0+5.5
AutomationBench 1.0.6 (Zapier)Agents17.122.4+5.3
HiL-Bench (Human-in-Loop Benchmark)Agents56.361.5+5.2
Vision Arena OverallMultimodal13091289−5.2
Agent Red Teaming k=100 (CAIS)Safety39.434.2+5.1
ArXivMath 04/2026Math72.577.5+5.0
ProofBench v1.1Math95.0100.0+5.0
NoCoT-Bench NCRI 15.2General QA119128+4.9
Clocktower BenchGames15341567+4.7
APEX-SWECode58.963.6+4.7
KernelBench Mega Normalized (RTX PRO 6000)Code69.464.7−4.7
LMArena Agent Arena Confirmed SuccessAgents5.114.4+4.7
ReactBench (39 tasks)Code44.649.2+4.6
APEX-AccountingFinance & Accounting56.461.0+4.6
LisanBench Path LengthGames1368015100+4.4
LiveBench 2.0Agents83.688.0+4.4
LMCAGeneral QA61.165.5+4.3
BrowseComp (Mercor 130-question subset)Search82.578.3−4.2
Hemingway-benchChat & Writing11101095−4.2
CadQueryEval (Corrected Geometry)Code80.076.0−4.0
DAYJOB: HealthcareHealth & Medicine5.69.6+4.0
SpatialViz-BenchMultimodal73.977.9+4.0
GBENCH One-shot Coding GScoreGames65.669.3+3.8
HardGeoBench <=100miMultimodal56.352.5−3.7
SWE-bench MultilingualCode94.891.1−3.7
Artificial Analysis Intelligence Index v4.3Indexes49.753.4+3.7
MineBench — Bradley-Terry RatingGames18691928+3.6
LMArena Agent Arena Steering BurdenAgents7.914.8+3.5
GDPval Extended (Mercor held-out rubric score)Professional Work65.368.7+3.4
SAGEMath51.948.5−3.4
FrontierCUA pass@1Agents40.036.7−3.3
Blueprint-Bench 2Multimodal0.40.4+3.3
FrontierMath Tiers 1-3 (v2) - PrivateMath87.090.2+3.2
Conceptual Reasoning IndexIndexes72.774.9+3.1
Excel Modeling BenchmarkFinance & Accounting73.776.7+3.0
Eval Connections One-Shot with TrapsGames85.082.0−3.0
Roboflow Vision Evals (October 2026, five tasks)Multimodal71.774.7+3.0
ChessBench EloGames10461095+2.9
GDPval (Mercor 213-task rubric score)Professional Work83.886.6+2.8
Extended NYT Connections 940 (Quadratic v1)Games92.790.0−2.7
SciCode (main problems)Science21.524.2+2.7
Terminal-Bench 2.1 Extended (Mercor held-out)Code16.819.5+2.7
HardGeoBench Avg DistanceMultimodal800888−2.7
APEX-Agents 1.1Professional Work74.577.1+2.6
HANDBOOK.mdProfessional Work36.238.8+2.6
Roboflow Vision Evals (September 2026 protocol)Multimodal78.781.3+2.6
MedCodeHealth & Medicine56.153.5−2.6
ERQAMultimodal67.870.3+2.5
HieroglyphBenchMultimodal22.920.4−2.5
FrontierMath Tier 4 (v2) - PrivateMath90.287.8−2.4
GeoBenchMultimodal44044303−2.0
Concept Synth INDUCTION Challenge100 Round 1General QA31.033.0+2.0
SWE Atlas - RefactoringCode54.856.7+1.9
AA-LCR Extended (Mercor held-out)Search71.173.0+1.9
MortgageTax (Vals all 1,258 documents)Finance & Accounting68.970.8+1.9
FrontierCode 1.1 Extended - Pass RateCode69.868.2−1.7
LLM Debate BenchmarkChat & Writing17271738+1.6
FrontierCode 1.1 Extended - ScoreCode64.362.7−1.6
LMArena Agent Arena SteerabilityAgents11.48.4−1.5
CharXiv Extended (Mercor held-out)Multimodal80.782.2+1.5
MindCube TinyMultimodal83.184.5+1.4
ACCoRDGeneral QA85.083.7−1.3
Ruby LLM Benchmarks — Program FixerCode77.276.0−1.3
HardGeoBench <=10miMultimodal41.340.0−1.2
SWE-bench VerifiedCode95.596.6+1.1
VCT-Refusal (CAIS)Safety100.098.9−1.1
LMArena Agent Arena Bash RecoveryAgents9.611.6+1.0
ARC-AGI-1 Semi-PrivateGeneral QA98.597.5−1.0
Chess Puzzles (task version 2.0)Games47.048.0+1.0
TaxEval v2Finance & Accounting76.976.0−1.0
WeirdML (v2)Science91.992.9+1.0
ARC-AGI-2General QA89.290.0+0.8
DTBenchGeneral QA98.497.6−0.8
LiveCodeBench (Vals AI public dataset)Code89.890.5+0.7
IntPhys 2Multimodal67.167.7+0.6
TextQuests Game ProgressGames56.156.6+0.5
BioTIER RefusalSafety100.099.5−0.5
SWE-TogetherCode68.869.3+0.5
Harvey LAB-AA v1.0 (criterion pass rate)Legal93.693.3−0.3
LMArena Text Arena EloChat & Writing15041503−0.2
Toloka Arena (v1)Agents60.960.7−0.2
LMArena Agent Arena Tool HallucinationAgents0.40.8+0.2
LMArena Agent Arena Bash ReliabilityAgents10.410.1−0.2
GPQA DiamondGeneral QA94.193.9−0.2
Prophet Arena Arena Score (Fixed Context)Forecasting84.985.0+0.1
LisanBench Difficulty-WeightedGames45624573+0.1
NoCoT-Bench NCKI (kspine v3)General QA128128+0.1
LegalBenchLegal88.688.5−0.1
KernelBench CUDA Normalized (RTX PRO 6000)Code51.651.60.0
AIME 2025Math100.0100.00.0
ArXivMath 05/2026Math87.587.50.0
BabaIsBench Stage 0: The IntroGames100.0100.00.0
FrontierMath ErdősMath0.00.00.0
HardGeoBench Country AccuracyMultimodal61.361.30.0
HPCT Refusal (CAIS)Safety100.0100.00.0
ObviousBenchGeneral QA99.399.30.0
OTIS Mock AIME 2024-2025Math100.0100.00.0
Senior SWE-BenchCode34.734.70.0

Claude Fable 5vsClaude Fable 5.1normalised score marginsData Nov 15, 2026Data: Mercor, Arena, Center for AI Safety and 53 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…