Atlas

Compare Models

Claude Fable 5 vs GPT-6 Sol

GPT-6 Sol leads by 0.3 capability points (≈95% interval −2.1 to +2.7): statistically tied · P(higher) ≈ 59%

Wins 58 of 114 shared benchmarks (53 lost, 3 tied)

Capability vs. relative cost
Claude Fable 5GPT-6 SolClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Fable 5vsGPT-6 Solcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins53 – 58 · 114 shared, 3 tied

Comparing matching setups only (107 benchmarks excluded) · suites counted once
← Claude Fable 5GPT-6 Sol →

Claude Fable 5vsGPT-6 Solwins by categoryData Nov 15, 2026Data: Mercor, Arena, Epoch AI and 36 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6 Sol minus Claude Fable 5.

KernelBench Mega Normalized (RTX PRO 6000)Code69.42.7−66.7
Vending-Bench 2Games453014428+59.9
Medical Long Context Reasoning (MLCR-AA)Health & Medicine64.416.1−48.3
BrokenArXiv 06/2026Math47.895.4+47.5
Search ArenaSearch12281174−43.0
BrokenArXiv 05/2026Math44.587.0+42.5
SWE-bench Verified Extended (Mercor held-out)Code78.739.3−39.4
SWE Atlas Codebase QnA Extended (Mercor held-out)Code0.031.3+31.3
EnterpriseBench: CoreCraftProfessional Work70.343.1−27.2
Terminal-Bench 2.1 Extended (Mercor held-out)Code16.842.4+25.6
Furniture Assembly BenchmarkMultimodal35.858.3+22.5
VoxelBench Text RatingGames21012394+20.9
CadQueryEval (Corrected Geometry)Code80.0100.0+20.0
FrontierCUA pass@1Agents40.020.0−20.0
ChartographyMultimodal34.853.6+18.8
YC-BenchGames19775731441793−18.7
BrowseComp Extended (Mercor held-out)Search44.563.2+18.7
Terminal-Bench Science 0.1 (Vals)Science12.931.4+18.6
GlobeBenchGeneral QA7.325.0+17.7
ZeroBench (Main questions, pass@5)Multimodal24.041.0+17.0
LisanBench Path LengthGames136808222−17.0
Tax Agent BenchFinance & Accounting69.853.0−16.8
AutomationBench 1.0.6 (Zapier)Agents17.133.2+16.2
AdvancedIFChat & Writing65.280.9+15.7
LisanBench Difficulty-WeightedGames45623118−15.5
ChessBench EloGames10461303+15.3
SWE-bench VerifiedCode95.580.3−15.2
MMMU-Pro Extended (Mercor held-out)Multimodal37.651.9+14.3
APEX-SWECode58.945.0−13.9
SWE-bench MultilingualCode94.881.7−13.1
ZeroBench Main Mean pass@1 (five samples)Multimodal15.828.8+13.0
ComplexConstraintsChat & Writing38.150.8+12.7
ProofBench v1.1Math95.083.0−12.0
GeneBench-Pro Public Release SubsetScience14.026.0+12.0
Olam Social Poker EloGames15531542−12.0
MedXpertQA MM Extended (Mercor held-out)Health & Medicine56.968.0+11.1
SWE Atlas - Codebase QnACode33.144.1+11.0
MMMU ProMultimodal72.782.5+9.8
KernelBench CUDA Normalized (RTX PRO 6000)Code51.642.0−9.6
MMLU ProGeneral QA79.488.9+9.5
Chess Puzzles (task version 2.0)Games47.056.0+9.0
MedCodeHealth & Medicine56.147.1−9.0
SciCode (main problems)Science21.512.7−8.8
SUPERChemScience64.472.8+8.4
LMArena Text Arena EloChat & Writing15041457−8.2
SWE-TogetherCode68.860.6−8.2
ZeroBench Main Questions pass^5Multimodal9.017.0+8.0
SAGEMath51.944.8−7.1
BrowseComp (Mercor 130-question subset)Search82.589.4+6.9
GDPval Extended (Mercor held-out rubric score)Professional Work65.358.7−6.6
NoCoT-Bench NCKI (kspine v3)General QA128119−6.6
Terminal-Bench-Science 0.1Science21.427.6+6.2
ALE-BenchCode24632637+6.1
CharXiv (descriptive and reasoning combined)Multimodal89.195.0+5.9
APEX-Agents 1.1Professional Work74.568.7−5.8
ChessBench CoherenceGames82.688.2+5.6
ArXivMath 06/2026Math85.491.0+5.5
Humanity's Last Exam (Text-Only)General QA51.546.3−5.2
Roboflow Vision Evals (October 2026, five tasks)Multimodal71.776.9+5.2
MedXpertQA MM (200-question set)Health & Medicine76.281.3+5.1
RiemannBenchMath60.064.8+4.8
GBENCH One-shot Coding GScoreGames65.670.1+4.5
GameBench 2Games75.279.4+4.2
Mystery Game PuzzlesGames52.056.0+4.0
Eval Connections One-Shot with TrapsGames85.089.0+4.0
Roboflow Vision Evals (September 2026 protocol)Multimodal78.782.3+3.6
Humanity's Sixth SenseMultimodal34.531.2−3.3
AA-Briefcase v1.1 EloAgents15361476−3.3
HieroglyphBenchMultimodal22.926.0+3.1
CharXiv Extended (Mercor held-out)Multimodal80.783.8+3.1
ARC-AGI-1 Semi-PrivateGeneral QA98.595.5−3.0
FrontierMath Tiers 1-3 (v2) - PrivateMath87.089.8+2.8
LMArena Agent Arena Net ImprovementAgents8.08.8+2.7
Extended NYT Connections 940 (Quadratic v1)Games92.790.1−2.6
Terminal-Bench 2.1Code88.085.4−2.6
ACCoRDGeneral QA85.082.4−2.6
Vibe Code Bench v1.1Code90.487.8−2.5
ArXivMath 05/2026Math87.590.0+2.5
NoCoT-Bench NCRI 15.2General QA119115−2.4
APEX-1 (Mercor, mean score)Professional Work66.063.6−2.4
ProgramBench (Mercor 100-task subset)Code0.02.3+2.3
Conceptual Reasoning IndexIndexes72.771.0−2.2
LMArena Agent Arena Praise vs ComplaintAgents14.310.0−2.1
Excel Modeling BenchmarkFinance & Accounting73.771.5−2.1
LMCAGeneral QA61.159.1−2.1
BullshitBench v2 LeaderboardSafety56.054.0−2.0
DAYJOB: HealthcareHealth & Medicine5.63.6−2.0
TaxCalcBench TY25Finance & Accounting30.028.0−2.0
Ruby LLM Benchmarks — Program FixerCode77.279.0+1.8
Blueprint-Bench 2Multimodal0.40.4−1.7
LiveBench 2.0Agents83.681.9−1.7
LMArena Agent Arena SteerabilityAgents11.414.5+1.6
Terminal-Bench 4.0 (Artificial Analysis)Code42.443.9+1.5
FrontierMath ErdősMath0.01.5+1.5
APEX-Accounting 1.1Finance & Accounting61.059.7−1.3
LMArena Agent Arena Bash RecoveryAgents9.67.0−1.3
LMArena Agent Arena Steering BurdenAgents7.910.1+1.1
DTBenchGeneral QA98.497.3−1.1
OpenAI internal computer-use safety - adverse outcome rateSafety18.317.4+0.9
AA-LCR Extended (Mercor held-out)Search71.170.2−0.9
Terminal-Bench 4.0Code44.543.8−0.7
LMArena Agent Arena Net Improvement (Steering Burden)Agents7.36.9−0.7
ARC-AGI-2General QA89.289.6+0.4
Humanity's Last Exam DiamondGeneral QA45.645.2−0.4
ChessBench AccuracyGames77.177.4+0.3
DAYJOB: FinanceFinance & Accounting9.09.3+0.3
GDPval (Mercor 213-task rubric score)Professional Work83.883.5−0.3
LMArena Agent Arena Tool HallucinationAgents0.41.0+0.3
LMArena Agent Arena Confirmed SuccessAgents5.14.5−0.3
LMArena Agent Arena Bash ReliabilityAgents10.410.9+0.3
FrontierMath Tier 4 (v2) - PrivateMath90.290.0−0.2
AIME 2025Math100.0100.00.0
ObviousBenchGeneral QA99.399.30.0
OTIS Mock AIME 2024-2025Math100.0100.00.0

Claude Fable 5vsGPT-6 Solnormalised score marginsData Nov 15, 2026Data: Mercor, Arena, Epoch AI and 36 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…