Atlas

Compare Models

Claude Fable 5.1 vs Grok 4.7

Claude Fable 5.1 leads by 5.1 capability points (≈95% interval +2.8 to +7.4): a clear lead over Grok 4.7 · P(higher) ≈ 100%

Wins 189 of 219 shared benchmarks (24 lost, 6 tied)

Capability vs. relative cost
Claude Fable 5.1Grok 4.7Closed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Fable 5.1vsGrok 4.7capability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins189 – 24 · 219 shared, 6 tied

Comparing matching setups only (13 benchmarks excluded) · suites counted once
← Claude Fable 5.1Grok 4.7 →

Claude Fable 5.1vsGrok 4.7wins by categoryData Nov 15, 2026Data: Mercor, Vals AI, Artificial Analysis and 68 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Grok 4.7 minus Claude Fable 5.1.

ProofBench v1.1Math100.026.0−74.0
FrontierMath Tier 4 (v2) - PrivateMath87.817.1−70.7
AA-PersonalityBench Informativeness indexChat & Writing96.233.4−63.5
BioSecBench-Refusal V1 Clean RerunsSafety0.062.4+62.4
Medical Long Context Reasoning (MLCR-AA)Health & Medicine71.115.0−56.1
AA-PersonalityBench Imagination indexChat & Writing86.735.3−51.9
AA-PersonalityBench Warmth indexChat & Writing94.344.8−50.0
Furniture Assembly BenchmarkMultimodal70.020.8−49.2
KernelBench Mega Normalized (RTX PRO 6000)Code64.718.0−46.7
BrokenArXiv 08/2026 (Native Harnesses)Math79.835.1−44.6
Design Creativity BenchCode48.662.2+43.1
ArXivMath 08/2026 (Native Harnesses)Math87.746.5−41.2
AA-PersonalityBench Transparency indexChat & Writing98.158.1−40.4
CAD ArenaAgents66.225.8−40.4
Toloka Arena (v3)Agents71.031.0−40.0
HardGeoBench Avg DistanceMultimodal8452104−38.8
BrokenArXiv 05/2026Math78.040.0−38.0
LLM Creative Story WritingChat & Writing3.80.4−37.6
FrontierMath Tiers 1-3 (v2) - PrivateMath90.253.0−37.2
ALE-BenchCode28491823−36.3
DeepSWE v1.1Code67.333.3−34.0
TaxCalcBench TY25Finance & Accounting42.08.0−34.0
BrokenArXiv 06/2026Math86.652.8−33.8
LMArena Agent Arena Net ImprovementAgents14.14.0−33.1
CheatBenchSafety45.878.0−32.2
ChartographyMultimodal46.214.7−31.5
FrontierML ($200 wallet)Agents41.910.9−31.0
AA-PersonalityBench Organization indexChat & Writing60.029.6−30.8
Agent Initiative BenchmarkAgents7.54.5−29.9
Arena Image-to-WebDevCode17241589−29.4
Mystery Game PuzzlesGames58.029.0−29.0
ARC-AGI-2General QA90.061.4−28.6
HardGeoBench <=10miMultimodal45.017.5−27.5
ArXivMath 05/2026Math87.560.8−26.7
HardGeoBench <=100miMultimodal53.827.5−26.2
BuseyBench SVGMultimodal7.04.6−24.6
ChessBench CoherenceGames61.986.3+24.4
HoneyBench v0.2 (hack rate)Safety61.885.1−23.3
LMArena Agent Arena Net Improvement (Steering Burden)Agents14.82.6−23.2
AA-PersonalityBench Expressiveness indexChat & Writing98.175.3−23.1
Terminal-Bench Science 0.1 (Vals)Science34.311.4−22.9
SWE Atlas Codebase QnA Extended (Mercor held-out)Code38.015.3−22.7
MysteryMechanismScience47.725.2−22.5
OpenWorldEngineer (Mean@8)Code29.17.0−22.1
CritPt v2 (agentic, avg@5)Science77.755.7−22.0
GameBench 2Games94.072.3−21.7
Toloka Arena (v1)Agents60.739.3−21.4
τ³-Banking (Mercor run, bm25_grep retrieval)Agents54.032.6−21.4
AA-3D v1.0 EloCode1139992−21.3
ArXivMath 06/2026Math91.070.1−20.8
Terminal-Bench 4.0Code57.937.6−20.3
DAYJOB: Data ScienceProfessional Work26.06.7−19.3
DistressBenchSafety71.552.4−19.2
Real-SWE 1.1Code45.628.0−17.6
SWE-bench VerifiedCode96.679.1−17.5
AA-PersonalityBench Conviction indexChat & Writing10082.9−17.3
ReactBench (39 tasks)Code49.232.3−16.9
LLM Debate BenchmarkChat & Writing17381620−16.8
FrontierCUA pass@1Agents36.720.0−16.7
Conceptual Reasoning IndexIndexes74.963.1−16.3
Code2Video-AA EloCode13251127−16.1
LMCAGeneral QA65.549.4−16.1
GeneBench-Pro Public Release SubsetScience26.010.0−16.0
Vals RSI Index v1.1 (four tasks)Agents36.120.2−15.9
Visa Agentic Commerce BenchmarkAgents75.660.0−15.6
sudo L7Code31.115.6−15.5
AA-PersonalityBench Slop indexChat & Writing23.88.6−15.4
Olam Social Poker EloGames14941508+15.2
HardGeoBench Country AccuracyMultimodal58.843.8−15.0
SimpleQA Verified (Epoch anti-abstention)General QA70.856.0−14.8
MedXpertQA MM (200-question set)Health & Medicine83.769.0−14.7
APEX-AccountingFinance & Accounting61.046.7−14.3
LMArena Agent Arena Praise vs ComplaintAgents31.63.1−14.3
MedXpertQA MM Extended (Mercor held-out)Health & Medicine66.052.0−14.0
LiveBench 2.0Agents88.074.1−13.9
RiemannBench (November 2026 rescoring)Math71.257.6−13.6
Bug Hunt BenchCode41.027.6−13.3
LMArena Code Arena WebDevCode17421641−13.2
Extended NYT Connections 940 (Quadratic v1)Games90.076.8−13.2
CritPt (avg@5)Science31.118.0−13.1
GBENCH One-shot Coding GScoreGames69.356.3−13.0
CadQueryEval (Corrected Geometry)Code76.088.0+12.0
SWE Atlas v1.1 - OverallCode74.162.1−12.0
DeepFinanceFinance & Accounting45.634.1−11.5
Humanity's Last Exam (Text-Only)General QA54.643.1−11.5
ProgramBench (Mercor 100-task subset)Code11.30.0−11.3
RuneBenchGames6.15.4−11.3
Roboflow Vision Evals (October 2026, five tasks)Multimodal74.763.5−11.2
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting79.768.6−11.2
Westworld Finance Diligence BenchFinance & Accounting58.247.2−11.1
LMArena Text Arena EloChat & Writing15031442−10.6
Web Clone BenchCode13.32.8−10.4
Vision Arena OverallMultimodal12891249−10.4
LegalReasonBenchLegal50.640.4−10.2
APEX-SWECode63.653.4−10.2
Design Arena Overall Code CategoriesCode13311270−10.1
CWE-bench v1 Programmatic Pass@1Cybersecurity58.068.0+10.0
RSI-Exam 0.1 (Full 88)Agents0.50.4−9.7
HealthBenchHealth & Medicine63.954.2−9.7
Excel Modeling BenchmarkFinance & Accounting76.767.0−9.7
ACCoRDGeneral QA83.774.0−9.6
Blueprint-Bench 2Multimodal0.40.3−9.4
Roboflow Vision Evals (September 2026 protocol)Multimodal81.371.9−9.4
EnterpriseBench: CoreCraftProfessional Work77.468.2−9.2
SpeciEval (Refusal-Aware Retries, Historical)Safety87.296.4+9.2
SpeciEval (Pooled English Runs)Safety87.296.4+9.2
Concept Synth INDUCTION Challenge100 Round 1General QA33.024.0−9.0
Epoch Capabilities Index (ECI)Indexes167156−8.9
Tax Agent BenchFinance & Accounting77.969.0−8.9
LMArena Agent Arena Steering BurdenAgents14.8-2.3−8.5
Snorkel SWE-bench CLICode14.56.1−8.4
APEX-1 (Mercor, mean score)Professional Work73.865.4−8.4
APEX-Agents 1.1Professional Work77.168.7−8.4
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting77.068.7−8.2
Finance Agent v2 - AdjustmentsFinance & Accounting54.446.3−8.1
Ruby LLM Benchmarks — Program FixerCode76.068.0−8.0
AutoResearchExam (hidden-test AUARC)Agents0.60.5−8.0
MMMU ProMultimodal86.178.2−7.9
SAGEMath48.540.8−7.7
Finance Agent v2 - Market AnalysisFinance & Accounting69.561.8−7.7
CVDP (517-problem set)Code62.654.9−7.7
Snorkel Agentic Coding 2.0Code39.631.9−7.7
Terminal-Bench 2.1 Extended (Mercor held-out)Code19.511.8−7.7
Finance Agent v2 - Financial ModelingFinance & Accounting25.317.7−7.7
AA-LCR v1.1Search87.580.0−7.5
ITBench-AAAgents49.542.1−7.4
Immigration BenchLegal75.468.0−7.4
Clipping Bench v1.0Games1068384−7.4
ARC-AGI-1 Semi-PrivateGeneral QA97.590.2−7.3
Senior SWE-BenchCode34.727.4−7.3
Creative Writing v3 EloChat & Writing21552012−7.2
DAYJOB: AccountingFinance & Accounting27.220.0−7.2
FrontierPhysics v0.1Science11.34.2−7.1
RWE-benchAgents7.00.0−7.0
ATLAS-Finance v1.0Finance & Accounting12.05.0−7.0
Chess Puzzles (task version 2.0)Games48.041.0−7.0
SWE-bench MultilingualCode91.184.2−6.9
SnorkelUnderwrite 2.0Finance & Accounting24.030.5+6.5
KernelBench CUDA Normalized (RTX PRO 6000)Code51.645.2−6.5
Finance Agent v2 - PrecedentsFinance & Accounting32.225.9−6.3
DeepResearch Bench II (Mercor 130-task set)Search55.049.0−6.0
SWE-bench Verified Extended (Mercor held-out)Code61.355.3−6.0
Artificial Analysis Coding Agent Index v1.5Indexes62.256.3−6.0
Finance Agent v2 - Disclosure AnalysisFinance & Accounting70.364.5−5.8
Finance Agent v2 - All PassFinance & Accounting47.842.1−5.6
Robo-benchAgents5.80.2−5.6
Ramp Accounting BenchFinance & Accounting49.744.1−5.6
CursorBench 4.0Code51.846.3−5.5
Biopharma Bench V0.1Health & Medicine69.964.4−5.5
SciCode 1.0.1 (Artificial Analysis)Code63.157.8−5.3
FrontierCode 2.0 - Pass RateCode37.632.3−5.3
DAYJOB: FinanceFinance & Accounting19.814.5−5.3
Excel Modeling Benchmark v1.1Finance & Accounting76.771.5−5.2
FrontierCode 2.0 - ScoreCode34.028.9−5.1
MentalHealthBenchSafety46.441.3−5.1
MineBench — Bradley-Terry RatingGames19281848−4.8
AA-LCR Extended (Mercor held-out)Search73.068.2−4.8
LMArena Agent Arena Confirmed SuccessAgents14.45.1−4.7
GDPxProfessional Work8.53.9−4.7
DAYJOB: TaxFinance & Accounting9.65.0−4.6
Snorkel Digital Work IndexIndexes23.619.3−4.3
FrontierCode 1.1 Extended - ScoreCode63.659.4−4.2
MMLU ProGeneral QA93.088.8−4.2
SWE-TogetherCode69.365.1−4.2
LegalBenchLegal88.584.4−4.1
CloudOpsBench v1.0Agents60.656.5−4.1
AdvancedIFChat & Writing84.280.2−4.0
CharXiv Extended (Mercor held-out)Multimodal82.278.2−4.0
LMArena Agent Arena SteerabilityAgents8.40.5−4.0
MedCodeHealth & Medicine53.549.6−4.0
ComplexConstraintsChat & Writing45.148.9+3.8
MissionReportProfessional Work63.359.5−3.8
Finance Agent v2 - ComparablesFinance & Accounting46.042.2−3.8
SWE Atlas - Codebase QnACode43.347.0+3.7
FrontierCode 1.1 Extended - Pass RateCode68.865.2−3.6
Terminal-Bench 2.1Code81.685.0+3.4
Realm Financial ReasoningFinance & Accounting47.844.5−3.3
HANDBOOK.md Agents v1.1Professional Work47.144.0−3.1
SnorkelLegalLegal35.932.8−3.1
CharXiv (descriptive and reasoning combined)Multimodal95.192.0−3.1
Harvey LAB-AA v1.1 (hallucination-gated)Legal6.49.4+3.0
DualEntry Accounting AI BenchmarkFinance & Accounting85.188.1+3.0
Eval Connections One-Shot with TrapsGames82.085.0+3.0
WellnessBenchSafety51.048.0−3.0
ChessBench AccuracyGames71.374.0+2.9
AA-Briefcase v1.1 EloAgents15771629+2.9
SM BenchSafety89.086.3−2.7
SnorkelRevOpsProfessional Work14.011.3−2.7
Arena Alignment IndexSafety80.182.7+2.6
BrowseComp (Mercor 130-question subset)Search78.375.8−2.5
AA-AnalystAgent (pass^5)Professional Work57.555.0−2.5
Longform Creative WritingChat & Writing85.383.0−2.3
LMArena Agent Arena Bash RecoveryAgents11.67.2−2.2
Harvey LAB (Mercor run, undisclosed ~213-task cohort)Legal17.419.5+2.1
OTIS Mock AIME 2024-2025Math100.098.1−1.9
MedScribeHealth & Medicine91.389.4−1.9
Financial Coaching BenchFinance & Accounting75.774.1−1.6
Creative Writing v3 Rubric ScoreChat & Writing16.315.9−1.6
DTBenchGeneral QA97.696.0−1.6
DAYJOB: HealthcareHealth & Medicine9.68.4−1.2
HealthBench (length-adjusted, Artificial Analysis)Health & Medicine57.256.0−1.2
ChessBench EloGames10951076−1.1
DAYJOB: LawLegal26.527.6+1.1
DAXBench v2Code91.592.6+1.1
GDPval-AA v2.1Professional Work17411717−1.0
MazeBench ASCIIGames2.01.0−1.0
Finance Agent v2 - Earnings AnalysisFinance & Accounting75.574.6−0.9
GDP.xlsxProfessional Work23.122.3−0.8
SnorkelManufacturingProfessional Work11.312.1+0.8
Snorkel WorkplaceAgentsProfessional Work15.815.2−0.6
SnorkelFinance 2.0Finance & Accounting19.620.2+0.6
LMArena Agent Arena Bash ReliabilityAgents10.19.3−0.4
LMArena Agent Arena Tool HallucinationAgents0.81.0+0.1
AIME 2025Math100.0100.00.0
Concept Synth INDUCTION Challenge64 Round 1General QA34.434.40.0
Dual-Arm Sim (task completion)Agents0.00.00.0
FrontierMath ErdősMath0.00.00.0
ObviousBenchGeneral QA99.399.30.0
SWE-Marathon v1.1Code45.645.60.0

Claude Fable 5.1vsGrok 4.7normalised score marginsData Nov 15, 2026Data: Mercor, Vals AI, Artificial Analysis and 68 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…