Atlas

Compare Models

Claude Fable 5.1 vs GPT-6 Sol

Claude Fable 5.1 leads by 2.0 capability points (≈95% interval −0.5 to +4.4): statistically tied · P(higher) ≈ 94%

Wins 126 of 181 shared benchmarks (49 lost, 6 tied)

Capability vs. relative cost
Claude Fable 5.1GPT-6 SolClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Fable 5.1vsGPT-6 Solcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins126 – 49 · 181 shared, 6 tied

Comparing matching setups only (84 benchmarks excluded) · suites counted once
← Claude Fable 5.1GPT-6 Sol →

Claude Fable 5.1vsGPT-6 Solwins by categoryData Nov 15, 2026Data: Mercor, Artificial Analysis, Vals AI and 50 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6 Sol minus Claude Fable 5.1.

AA-PersonalityBench Informativeness indexChat & Writing96.210.5−86.5
AA-PersonalityBench Expressiveness indexChat & Writing98.133.4−65.4
KernelBench Mega Normalized (RTX PRO 6000)Code64.72.7−62.0
AA-PersonalityBench Transparency indexChat & Writing98.139.1−59.6
AA-PersonalityBench Imagination indexChat & Writing86.729.6−57.7
Medical Long Context Reasoning (MLCR-AA)Health & Medicine71.116.1−55.0
Olam Social Poker EloGames14941542+52.2
Search ArenaSearch12371174−50.4
AA-PersonalityBench Conviction indexChat & Writing10050.5−50.0
AA-PersonalityBench Organization indexChat & Writing60.012.4−48.1
AA-PersonalityBench Warmth indexChat & Writing94.354.3−40.4
EnterpriseBench: CoreCraftProfessional Work77.443.1−34.3
BioSecBench-Refusal V1 Clean RerunsSafety5.637.9+32.3
ChessBench CoherenceGames61.988.2+26.3
Terminal-Bench-Science 0.1Science52.627.6−25.0
Tax Agent BenchFinance & Accounting77.953.0−24.8
CadQueryEval (Corrected Geometry)Code76.0100.0+24.0
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting77.053.6−23.3
AA-PersonalityBench Slop indexChat & Writing23.81.0−23.1
SM BenchSafety89.066.0−23.0
BullshitBench v2 LeaderboardSafety77.054.0−23.0
Terminal-Bench 2.1 Extended (Mercor held-out)Code19.542.4+22.9
SWE-bench Verified Extended (Mercor held-out)Code61.339.3−22.0
LisanBench Path LengthGames151008222−21.4
HardGeoBench Avg DistanceMultimodal8451497−20.1
HardGeoBench <=10miMultimodal45.026.3−18.7
LegalReasonBenchLegal50.631.9−18.7
APEX-SWECode63.645.0−18.6
MysteryMechanismScience47.730.2−17.6
Humanity's Last Exam DiamondGeneral QA51.333.8−17.5
LMArena Agent Arena Net ImprovementAgents14.18.8−17.4
ProofBench v1.1Math100.083.0−17.0
ChessBenchGames45.362.0+16.7
FrontierCUA pass@1Agents36.720.0−16.7
SWE-bench VerifiedCode96.680.3−16.3
HardGeoBench <=100miMultimodal53.837.5−16.2
HealthBenchHealth & Medicine63.948.2−15.6
LisanBench Difficulty-WeightedGames45733118−15.6
LMArena Agent Arena Net Improvement (Steering Burden)Agents14.86.9−14.9
Finance Agent v2 - Disclosure AnalysisFinance & Accounting70.355.6−14.6
GameBench 2Games94.079.4−14.6
Terminal-Bench 4.0Code57.943.8−14.1
TaxCalcBench TY25Finance & Accounting42.028.0−14.0
SimpleBenchGeneral QA86.673.1−13.5
Harvey LAB (Mercor run, undisclosed ~213-task cohort)Legal17.44.1−13.3
Financial Coaching BenchFinance & Accounting75.762.6−13.1
Toloka Arena (v3)Agents71.058.0−13.0
Professional Reasoning Benchmark - LegalLegal51.638.6−13.0
HardGeoBench Country AccuracyMultimodal58.846.3−12.5
ChessBench EloGames10951303+12.4
DeepResearch Bench II (Mercor 130-task set)Search55.042.9−12.1
Finance Agent v2 - All PassFinance & Accounting47.836.0−11.8
Furniture Assembly BenchmarkMultimodal70.058.3−11.7
Finance Agent v2 - Market AnalysisFinance & Accounting69.558.0−11.6
SciCode (main problems)Science24.212.7−11.5
SpeciEval (Pooled English Runs)Safety87.298.5+11.3
SpeciEval (Refusal-Aware Retries, Historical)Safety87.298.5+11.3
Finance Agent v2 - AdjustmentsFinance & Accounting54.443.2−11.2
BrowseComp (Mercor 130-question subset)Search78.389.4+11.1
GDPval-AA v2.1Professional Work17651514−11.0
τ³-Banking (Mercor run, bm25_grep retrieval)Agents54.043.0−11.0
LMArena Agent Arena Praise vs ComplaintAgents31.610.0−10.8
AutomationBench 1.0.6 (Zapier)Agents22.433.2+10.8
AA-Briefcase v1.1 EloAgents16711476−10.7
DAYJOB: FinanceFinance & Accounting19.89.3−10.5
APEX-1 (Mercor, mean score)Professional Work73.863.6−10.2
SimpleQA Verified (Epoch anti-abstention)General QA70.860.7−10.1
GDPval Extended (Mercor held-out rubric score)Professional Work68.758.7−10.0
CVDP (517-problem set)Code62.652.7−9.9
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting79.770.0−9.7
KernelBench CUDA Normalized (RTX PRO 6000)Code51.642.0−9.6
SWE-bench MultilingualCode91.181.7−9.4
MedScribeHealth & Medicine91.382.0−9.3
Gray Swan IPI Attack Success (k=15)Safety1.010.1−9.1
BrokenArXiv 05/2026Math78.087.0+9.0
LibraryDesignBenchCode47.538.5−9.0
ProgramBench (Mercor 100-task subset)Code11.32.3−9.0
DualEntry Accounting AI BenchmarkFinance & Accounting85.176.2−8.9
BrokenArXiv 06/2026Math86.695.4+8.8
SWE-TogetherCode69.360.6−8.7
APEX-Agents 1.1Professional Work77.168.7−8.4
Humanity's Last Exam (Text-Only)General QA54.646.3−8.3
LMArena Text Arena EloChat & Writing15031457−8.0
Chess Puzzles (task version 2.0)Games48.056.0+8.0
CritPt v2 (agentic, avg@5)Science72.980.9+8.0
HealthBench ProfessionalHealth & Medicine69.261.2−8.0
ALE-BenchCode28492637−7.5
MentalHealthBenchSafety46.453.9+7.5
BioSecBench-Structure V1 (58 evaluations)Science6.313.8+7.5
Arena Alignment IndexSafety80.187.6+7.5
ChartographyMultimodal46.253.6+7.4
NoCoT-Bench NCRI 15.2General QA128115−7.4
SpireClimbGames2.09.2+7.2
APEX-Accounting 1.1Finance & Accounting66.759.7−7.0
Eval Connections One-Shot with TrapsGames82.089.0+7.0
SWE Atlas Codebase QnA Extended (Mercor held-out)Code38.031.3−6.7
NoCoT-Bench NCKI (kspine v3)General QA128119−6.7
ChessBench AccuracyGames71.377.4+6.5
MedCodeHealth & Medicine53.547.1−6.4
LMCAGeneral QA65.559.1−6.4
Design Arena Overall Code CategoriesCode13311293−6.3
LiveBench 2.0Agents88.081.9−6.1
Finance Agent v2 - Financial ModelingFinance & Accounting25.319.3−6.0
GDP.xlsxProfessional Work23.117.1−6.0
DAYJOB: HealthcareHealth & Medicine9.63.6−6.0
ComplexConstraintsChat & Writing45.150.8+5.7
HieroglyphBenchMultimodal20.426.0+5.6
SciCode 1.0.1 (Artificial Analysis)Code63.157.6−5.4
Conceptual Reasoning IndexIndexes74.971.0−5.4
FrontierCode 2.0 (Artificial Analysis)Code30.335.6+5.3
ArXivMath 08/2026 (Native Harnesses)Math87.793.0+5.3
Finance Agent v2 - ComparablesFinance & Accounting46.040.8−5.1
Excel Modeling BenchmarkFinance & Accounting76.771.5−5.1
WellnessBenchSafety51.056.0+5.0
Blueprint-Bench 2Multimodal0.40.4−5.0
BuseyBench SVGMultimodal7.07.5+5.0
LMArena Agent Arena Confirmed SuccessAgents14.44.5−5.0
Qodo AI Code Review Benchmark (125 PRs)Code53.548.9−4.6
MMMU-Pro Extended (Mercor held-out)Multimodal47.651.9+4.3
MMLU ProGeneral QA93.088.9−4.1
AA-LCR v1.1Search87.583.7−3.8
EBR-bench v4 (card ban, single agent)Games57.153.3−3.8
Terminal-Bench 2.1Code81.685.4+3.8
SAGEMath48.544.8−3.7
Finance Agent v2 - PrecedentsFinance & Accounting32.228.5−3.7
DistressBenchSafety71.568.0−3.6
MMMU ProMultimodal86.182.5−3.6
GlobeBenchGeneral QA21.625.0+3.4
AdvancedIFChat & Writing84.280.9−3.3
GDPval (Mercor 213-task rubric score)Professional Work86.683.5−3.1
Finance Agent v2 - Earnings AnalysisFinance & Accounting75.572.4−3.1
Ruby LLM Benchmarks — Program FixerCode76.079.0+3.1
LMArena Agent Arena SteerabilityAgents8.414.5+3.1
HealthBench (length-adjusted, Artificial Analysis)Health & Medicine57.254.2−3.0
Terminal-Bench Science 0.1 (Vals)Science34.331.4−2.9
AA-LCR Extended (Mercor held-out)Search73.070.2−2.8
Harvey LAB-AA v1.1 (hallucination-gated)Legal6.43.6−2.8
ArXivMath 05/2026Math87.590.0+2.5
Longform Creative WritingChat & Writing85.382.8−2.5
Epoch Capabilities Index (ECI)Indexes167164−2.5
MedXpertQA MM (200-question set)Health & Medicine83.781.3−2.4
LMArena Agent Arena Steering BurdenAgents14.810.1−2.3
LMArena Agent Arena Bash RecoveryAgents11.67.0−2.3
Roboflow Vision Evals (October 2026, five tasks)Multimodal74.776.9+2.2
FrontierMath Tier 4 (v2) - PrivateMath87.890.0+2.2
ARC-AGI-1 Semi-PrivateGeneral QA97.595.5−2.0
MedXpertQA MM Extended (Mercor held-out)Health & Medicine66.068.0+2.0
Mystery Game PuzzlesGames58.056.0−2.0
SUPERChemScience74.772.8−1.9
Clipping Bench v1.0Games10681240+1.9
CharXiv Extended (Mercor held-out)Multimodal82.283.8+1.6
Creative Writing v3 EloChat & Writing21552126−1.5
FrontierMath ErdősMath0.01.5+1.5
ACCoRDGeneral QA83.782.4−1.3
SAB ResearchFinance & Accounting96.895.6−1.3
BrokenArXiv 08/2026 (Native Harnesses)Math79.878.6−1.2
AA-3D v1.0 EloCode11391131−1.1
Creative Writing v3 Rubric ScoreChat & Writing16.316.1−1.1
Roboflow Vision Evals (September 2026 protocol)Multimodal81.382.3+1.0
SWE Atlas - Codebase QnACode43.344.1+0.8
RiemannBenchMath65.664.8−0.8
GBENCH One-shot Coding GScoreGames69.370.1+0.8
DeepSWE v1.1Code67.366.6−0.7
SAB RankingForecasting-0.1-0.1+0.7
LMArena Agent Arena Bash ReliabilityAgents10.110.9+0.4
ARC-AGI-2General QA90.089.6−0.4
GDP.pdfProfessional Work27.628.0+0.4
FrontierMath Tiers 1-3 (v2) - PrivateMath90.289.8−0.4
CritPt (avg@5)Science31.130.9−0.3
DTBenchGeneral QA97.697.3−0.3
FORTRESS Average Risk Score (ARS)Safety13.713.5+0.3
LMArena Agent Arena Tool HallucinationAgents0.81.0+0.1
CharXiv (descriptive and reasoning combined)Multimodal95.195.0−0.1
Extended NYT Connections 940 (Quadratic v1)Games90.090.1+0.1
ITBench-AAAgents49.549.4−0.1
AIME 2025Math100.0100.00.0
ArXivMath 06/2026Math91.091.00.0
GeneBench-Pro Public Release SubsetScience26.026.00.0
Next.js AI Agent EvaluationsAgents100.0100.00.0
ObviousBenchGeneral QA99.399.30.0
OTIS Mock AIME 2024-2025Math100.0100.00.0

Claude Fable 5.1vsGPT-6 Solnormalised score marginsData Nov 15, 2026Data: Mercor, Artificial Analysis, Vals AI and 50 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…