Atlas

Compare Models

Claude Fable 5.1 vs DeepSeek V4.1 Flash

Claude Fable 5.1 leads by 6.3 capability points (≈95% interval +4.1 to +8.6): a clear lead over DeepSeek V4.1 Flash · P(higher) ≈ 100%

Wins 153 of 175 shared benchmarks (20 lost, 2 tied)

Capability vs. relative cost
Claude Fable 5.1DeepSeek V4.1 FlashClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Fable 5.1vsDeepSeek V4.1 Flashcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins153 – 20 · 175 shared, 2 tied

Comparing matching setups only (12 benchmarks excluded) · suites counted once
← Claude Fable 5.1DeepSeek V4.1 Flash →

Claude Fable 5.1vsDeepSeek V4.1 Flashwins by categoryData Nov 15, 2026Data: Mercor, Arena, Artificial Analysis and 45 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is DeepSeek V4.1 Flash minus Claude Fable 5.1.

FrontierMath Tier 4 (v2) - PrivateMath87.826.8−61.0
AA-PersonalityBench Imagination indexChat & Writing86.731.5−55.8
AA-PersonalityBench Warmth indexChat & Writing94.341.0−53.8
BrokenArXiv 05/2026Math78.028.5−49.5
Medical Long Context Reasoning (MLCR-AA)Health & Medicine71.122.8−48.3
LLM Creative Story WritingChat & Writing3.8-0.5−48.0
BrokenArXiv 06/2026Math86.639.4−47.2
ProofBench v1.1Math100.054.0−46.0
ArXivMath 08/2026 (Native Harnesses)Math87.743.9−43.9
RiemannBenchMath65.624.0−41.6
AA-PersonalityBench Organization indexChat & Writing60.0100+40.4
AA-PersonalityBench Transparency indexChat & Writing98.160.0−38.5
TaxCalcBench TY25Finance & Accounting42.04.0−38.0
SPU (Science Prediction Universe)Science-0.50.2+37.9
AA-PersonalityBench Expressiveness indexChat & Writing98.161.9−36.5
ALE-BenchCode28491829−36.1
EnterpriseBench: CoreCraftProfessional Work77.441.5−35.9
Furniture Assembly BenchmarkMultimodal70.034.2−35.8
ChartographyMultimodal46.211.6−34.6
LMArena Agent Arena Net ImprovementAgents14.13.9−33.7
ArXivMath 06/2026Math91.057.6−33.3
LMArena Agent Arena Chat Net Improvement (Steering Burden)Agents15.21.6−33.2
ArXivMath 04/2026Math77.545.0−32.5
Creative Writing v3 EloChat & Writing21551540−31.1
FrontierCUA pass@1Agents36.76.7−30.0
Terminal-Bench Science 0.1 (Vals)Science34.34.3−30.0
BuseyBench SVGMultimodal7.04.1−29.4
ArXivMath 05/2026Math87.558.3−29.2
SimpleQA Verified (Epoch anti-abstention)General QA70.842.9−27.9
LMArena Agent Arena Work Net Improvement (Steering Burden)Agents13.92.5−27.4
Terminal-Bench 4.0Code57.931.2−26.7
APEX-Agents 1.1Professional Work77.150.5−26.6
MysteryMechanismScience47.721.2−26.6
GameBench 2Games94.068.4−25.6
LMArena Agent Arena Code Net Improvement (Steering Burden)Agents13.92.7−25.3
BrokenArXiv 08/2026 (Native Harnesses)Math79.854.8−25.0
Arena Image-to-WebDevCode17241610−24.7
Humanity's Last ExamGeneral QA60.936.8−24.1
FrontierMath Tiers 1-3 (v2) - PrivateMath90.267.4−22.8
Conceptual Reasoning IndexIndexes74.958.4−22.7
MMMU-Pro Extended (Mercor held-out)Multimodal47.625.0−22.6
AA-AnalystAgent (pass^5)Professional Work57.535.0−22.5
LLM Debate BenchmarkChat & Writing17381582−22.1
Code2Video-AA EloCode13251053−22.1
CloudOpsBench v1.0Agents43.821.8−22.0
Concept Synth INDUCTION Challenge100 Round 1General QA33.011.0−22.0
SWE-bench Verified Extended (Mercor held-out)Code61.382.7+21.4
GBENCH One-shot Coding GScoreGames69.348.0−21.3
BioSecBench-Structure V1 (58 evaluations)Science6.327.6+21.3
ReactBench (39 tasks): Writing ReactCode67.847.0−20.9
Bug Hunt BenchCode41.020.6−20.3
MedXpertQA MM (200-question set)Health & Medicine83.763.5−20.2
WeirdML v3Science26.05.9−20.0
BullshitBench v2 LeaderboardSafety77.057.0−20.0
SimpleBenchGeneral QA86.666.7−19.9
Excel Modeling BenchmarkFinance & Accounting76.757.2−19.5
Terminal-Bench 2.1 Extended (Mercor held-out)Code19.538.4+18.9
LMCAGeneral QA65.547.0−18.5
SUPERChemScience74.756.7−18.0
MedXpertQA MM Extended (Mercor held-out)Health & Medicine66.048.4−17.6
AA-PersonalityBench Informativeness indexChat & Writing96.279.1−17.3
Concept Synth INDUCTION Challenge64 Round 1General QA34.417.2−17.2
ARC-AGI-2General QA90.072.9−17.1
CritPt (avg@5)Science31.114.3−16.9
OpenWorldEngineer (Mean@8)Code29.112.4−16.7
KernelBench Mega Normalized (RTX PRO 6000)Code64.748.2−16.5
HealthBench ProfessionalHealth & Medicine69.252.9−16.3
τ³-Banking (Mercor run, bm25_grep retrieval)Agents54.037.8−16.2
Design Creativity BenchCode48.643.6−16.1
DAXBench v2Code91.575.5−16.0
LMArena Code Arena WebDevCode17421621−15.9
ACCoRDGeneral QA83.768.1−15.6
Tax Agent BenchFinance & Accounting77.962.5−15.4
GeneBench-Pro Public Release SubsetScience26.011.0−15.0
Mystery Game PuzzlesGames58.043.0−15.0
LMArena Agent Arena Chat Praise vs ComplaintAgents32.03.1−14.5
AA-Briefcase EloAgents16621424−13.8
Artificial Analysis Intelligence Index v4.3Indexes53.439.5−13.8
SWE-bench VerifiedCode96.682.9−13.7
AA-PersonalityBench Slop indexChat & Writing23.837.2+13.5
TxBench-Antibody-Development V1 (60 evaluations)Science27.814.4−13.3
CharXiv Extended (Mercor held-out)Multimodal82.268.9−13.3
OpenDS-WorldSearch33.520.5−13.0
SciCode (main problems)Science24.211.2−13.0
APEX-AccountingFinance & Accounting61.048.1−12.9
MedCodeHealth & Medicine53.541.2−12.3
Senior SWE-BenchCode34.722.7−12.0
APEX-SWECode63.651.9−11.7
LMArena Agent Arena Work Praise vs ComplaintAgents26.93.7−11.6
Longform Creative WritingChat & Writing85.374.0−11.3
SciCode 1.0.1 (Artificial Analysis)Code63.151.9−11.2
Creative Writing v3 Rubric ScoreChat & Writing16.314.1−11.1
LMArena Agent Arena Code Praise vs ComplaintAgents23.53.0−10.2
SWE Atlas - Codebase QnACode43.353.5+10.2
SpeciEval (Refusal-Aware Retries, Historical)Safety87.297.4+10.1
SpeciEval (Pooled English Runs)Safety87.297.4+10.1
Westworld Finance Diligence BenchFinance & Accounting58.248.3−10.0
LiveBench 2.0Agents88.078.0−9.9
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting79.769.8−9.9
Immigration BenchLegal75.465.6−9.8
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting77.067.4−9.6
Eval Connections One-Shot with TrapsGames82.073.0−9.0
Terminal-Bench 2.1Code81.690.6+9.0
Excel Modeling Benchmark v1.1Finance & Accounting76.767.8−9.0
Finance Agent v2 - Disclosure AnalysisFinance & Accounting70.361.3−9.0
Ramp Accounting BenchFinance & Accounting49.740.8−8.9
LMArena Agent Arena Code Steering BurdenAgents19.62.3−8.6
Epoch Capabilities Index (ECI)Indexes167156−8.6
APEX-1 (Mercor, mean score)Professional Work73.865.5−8.3
DistressBenchSafety71.579.8+8.2
LMArena Agent Arena Work Steering BurdenAgents17.41.5−8.0
DTBenchGeneral QA97.689.9−7.7
LMArena Agent Arena Chat Bash ReliabilityAgents13.8-1.0−7.4
GDPval-AA v2.1Professional Work17651600−7.3
SWE-bench MultilingualCode91.198.2+7.1
FrontierCode 1.1 Extended - ScoreCode63.656.6−7.0
Vision Arena OverallMultimodal12891263−6.8
FrontierCode 1.1 Extended - Pass RateCode68.862.0−6.8
Finance Agent v2 - All PassFinance & Accounting47.841.1−6.7
LMArena Agent Arena Chat Confirmed SuccessAgents18.24.9−6.7
GDPval-AA v2Professional Work17641632−6.5
MineBench — Bradley-Terry RatingGames19281822−6.4
Finance Agent v2 - Market AnalysisFinance & Accounting69.563.4−6.2
LMArena Agent Arena SteerabilityAgents8.4-3.3−5.8
MedScribeHealth & Medicine91.385.5−5.8
BioSecBench-Refusal V1 Clean RerunsSafety5.611.3+5.7
LMArena Agent Arena Chat Steering BurdenAgents10.9-0.5−5.7
Arena Alignment IndexSafety80.174.8−5.3
DeepSWE v1.1Code67.372.6+5.3
LegalBenchLegal88.583.3−5.2
Finance Agent v2 - AdjustmentsFinance & Accounting54.449.2−5.2
LMArena Agent Arena Code Bash ReliabilityAgents9.9-0.2−5.0
CharXiv (descriptive and reasoning combined)Multimodal95.190.1−5.0
SWEeper-BenchCode56.551.5−5.0
LMArena Agent Arena Work Bash ReliabilityAgents10.00.1−4.9
RuneBenchGames6.05.7−4.9
LMArena Text Arena EloChat & Writing15031475−4.8
MMLU ProGeneral QA93.088.2−4.8
Harvey LAB-AA v1.1 (hallucination-gated)Legal6.41.7−4.7
KernelBench CUDA Normalized (RTX PRO 6000)Code51.647.0−4.6
ComplexConstraintsChat & Writing45.149.7+4.6
Finance Agent v2 - PrecedentsFinance & Accounting32.228.2−4.0
AA-PersonalityBench Conviction indexChat & Writing10096.2−3.8
ReactBench (39 tasks): Fixing ReactCode22.518.8−3.7
HealthBench (length-adjusted, Artificial Analysis)Health & Medicine57.260.9+3.7
LMArena Agent Arena Code Confirmed SuccessAgents13.96.6−3.7
LMArena Agent Arena Work Confirmed SuccessAgents12.45.7−3.4
BabyVisionMultimodal46.443.3−3.1
ARC-AGI-1 Semi-PrivateGeneral QA97.594.5−3.0
CWE-bench v1 Programmatic Pass@1Cybersecurity58.055.0−3.0
Financial Coaching BenchFinance & Accounting75.772.8−2.9
ITBench-AAAgents49.546.9−2.6
Finance Agent v2 - Earnings AnalysisFinance & Accounting75.572.9−2.6
Design Arena Overall Code CategoriesCode13311316−2.5
Finance Agent v2 - Financial ModelingFinance & Accounting25.323.1−2.3
AA-LCR Extended (Mercor held-out)Search73.070.8−2.2
HealthBenchHealth & Medicine63.961.9−2.0
LMArena Agent Arena Bash RecoveryAgents11.67.9−1.9
DeepResearch Bench II (Mercor 130-task set)Search55.053.2−1.8
AIME 2025Math100.098.3−1.7
OTIS Mock AIME 2024-2025Math100.098.3−1.7
BioSecBench-RespiratoryVirus-Surveillance V1 (110 evaluations)Science3.65.2+1.5
SWE Atlas Codebase QnA Extended (Mercor held-out)Code38.036.7−1.3
BrowseComp (Mercor 130-question subset)Search78.379.4+1.1
DualEntry Accounting AI BenchmarkFinance & Accounting85.184.2−0.9
SAGEMath48.547.9−0.6
Extended NYT Connections 940 (Quadratic v1)Games90.089.6−0.4
AdvancedIFChat & Writing84.284.0−0.2
GDPval (Mercor 213-task rubric score)Professional Work86.686.7+0.1
LMArena Agent Arena Work Tool HallucinationAgents0.70.5−0.1
LMArena Agent Arena Chat Tool HallucinationAgents0.50.3−0.1
Finance Agent v2 - ComparablesFinance & Accounting46.046.1+0.1
LMArena Agent Arena Code Tool HallucinationAgents0.70.5−0.1
CadQueryEval (Corrected Geometry)Code76.076.00.0
Chess Puzzles (task version 2.0)Games48.048.00.0

Claude Fable 5.1vsDeepSeek V4.1 Flashnormalised score marginsData Nov 15, 2026Data: Mercor, Arena, Artificial Analysis and 45 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…