Atlas

Compare Models

Claude Fable 5.1 vs GPT-6 Astra

GPT-6 Astra leads by 1.5 capability points (≈95% interval −1.1 to +4.1): statistically tied · P(higher) ≈ 87%

Wins 143 of 282 shared benchmarks (133 lost, 6 tied)

Capability vs. relative cost
Claude Fable 5.1GPT-6 AstraClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Fable 5.1vsGPT-6 Astracapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins133 – 143 · 282 shared, 6 tied

Comparing matching setups only (143 benchmarks excluded) · suites counted once
← Claude Fable 5.1GPT-6 Astra →

Claude Fable 5.1vsGPT-6 Astrawins by categoryData Nov 15, 2026Data: Mercor, Artificial Analysis, Vals AI and 81 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6 Astra minus Claude Fable 5.1.

Vending-Bench Arena: Round 12Games571912363+100.0
Clipping Bench v1.0Games10689514+90.9
VCT-Refusal (CAIS)Safety98.917.6−81.3
AA-PersonalityBench Informativeness indexChat & Writing96.229.6−67.3
Concept Synth INDUCTION Challenge64 Round 1General QA34.498.4+64.1
ActiveVisionMultimodal15.376.5+61.2
Concept Synth INDUCTION Challenge100 Round 1General QA33.094.0+61.0
Olam Social Poker EloGames14941550+60.9
AA-PersonalityBench Conviction indexChat & Writing10044.8−55.8
Design Creativity BenchCode48.665.5+53.4
HPCT Refusal (CAIS)Safety100.050.0−50.0
Clocktower BenchGames15671892+46.6
Document ArenaProfessional Work15151467−42.6
AA-PersonalityBench Expressiveness indexChat & Writing98.158.1−40.4
ChessBench EloGames10951755+39.3
Medical Long Context Reasoning (MLCR-AA)Health & Medicine71.135.0−36.1
BARC-Bench (all-safe rate)Safety37.372.7+35.5
BioSecBench-Refusal V1 Clean RerunsSafety5.640.8+35.2
ChessBench CoherenceGames61.997.0+35.1
EnterpriseBench: CoreCraftProfessional Work77.442.6−34.8
SpireClimbGames2.034.0+32.0
Terminal-Bench Science 0.1 (Vals)Science34.365.7+31.4
GlobeBenchGeneral QA21.649.6+27.9
Time Horizon Index: KSPAgents63.390.5+27.2
MirrorCode (ML, +Private, 2L)Code73.346.7−26.6
Mystery Game PuzzlesGames58.084.0+26.0
Chess Puzzles (task version 2.0)Games48.074.0+26.0
EnigmaEvalGeneral QA44.269.7+25.5
ChartographyMultimodal46.271.0+24.8
ChessBenchGames45.370.0+24.7
CadQueryEval (Corrected Geometry)Code76.0100.0+24.0
LisanBench Path LengthGames151007393−23.9
Agent Red Teaming k=100 (CAIS)Safety34.213.5+20.7
HieroglyphBenchMultimodal20.440.0+19.6
BioTIER RefusalSafety99.579.9−19.6
AA-PersonalityBench Warmth indexChat & Writing94.375.3−19.2
SM BenchSafety89.069.9−19.1
EBR-bench v4 (card ban, single agent)Games57.176.2+19.0
AutomationBench 1.0.6 (Zapier)Agents22.441.4+19.0
LLM Debate BenchmarkChat & Writing17381606−18.8
Machiavelli Deception (CAIS)Safety82.476.3+18.5
TextQuests Game ProgressGames56.674.2+17.6
Hedge-BenchFinance & Accounting26.844.2+17.4
NoCoT-Bench NCRI 15.2General QA128159+17.0
LABBench2Science68.685.4+16.8
BrowseComp (Mercor 130-question subset)Search78.394.2+15.9
HANDBOOK.md Agents v1.1Professional Work47.131.4−15.7
LisanBench Difficulty-WeightedGames45733113−15.6
ClockBench AccuracyMultimodal59.475.0+15.6
SpatialViz-BenchMultimodal77.993.1+15.2
OpenWorldEngineer (Mean@8)Code29.144.2+15.1
APEX-SWECode63.648.6−15.0
SnorkelLegalLegal35.920.9−15.0
BioSecBench-Structure V1 (58 evaluations)Science6.321.3+14.9
Agent Initiative BenchmarkAgents7.59.0+14.7
ChessBench AccuracyGames71.385.0+14.7
Tax Agent BenchFinance & Accounting77.963.3−14.5
ArXivMath 04/2026Math77.590.8+13.3
BrokenArXiv 05/2026Math78.091.0+13.0
WellnessBenchSafety51.064.0+13.0
SciUniverse Level 1Science45.332.5−12.9
ComplexConstraintsChat & Writing45.157.7+12.6
MASKSafety85.998.5+12.6
BrokenArXiv 06/2026Math86.699.1+12.5
MMMU-Pro Extended (Mercor held-out)Multimodal47.660.0+12.4
LMArena Agent Arena Net ImprovementAgents14.110.4−12.2
ERQAMultimodal70.382.3+12.0
Eval Connections One-Shot with TrapsGames82.094.0+12.0
Terminal-Bench-Science 0.1Science52.664.6+12.0
ALE-BenchCode28493164+11.2
Visa Agentic Commerce BenchmarkAgents75.664.4−11.1
LegalReasonBenchLegal50.639.5−11.1
Last Translation Benchmark (LTBv1-eval, blind)Chat & Writing32.944.0+11.1
HoneyBench v0.2 (hack rate)Safety61.850.8+11.1
MentalHealthBenchSafety46.457.3+10.9
DAYJOB: LawLegal26.536.9+10.4
Furniture Assembly BenchmarkMultimodal70.080.0+10.0
GeneBench-Pro Public Release SubsetScience26.036.0+10.0
GBENCH One-shot Coding GScoreGames69.379.2+9.9
FrontierCode 2.0 (Artificial Analysis)Code30.340.1+9.8
FrontierMath Tier 4 (v2) - PrivateMath87.897.6+9.8
IntPhys 2Multimodal67.777.4+9.7
BioSecBench-RespiratoryVirus-Surveillance V1 (110 evaluations)Science3.613.3+9.7
AA-PersonalityBench Organization indexChat & Writing60.069.5+9.6
AA-PersonalityBench Imagination indexChat & Writing86.777.2−9.6
AutomationBench-AA - release-reported strict pass rateAgents32.141.6+9.5
SpeciEval (Refusal-Aware Retries, Historical)Safety87.296.5+9.3
SpeciEval (Pooled English Runs)Safety87.296.5+9.3
Humanity's Last Exam DiamondGeneral QA51.360.6+9.3
GDPval-AA v2Professional Work17641580−9.0
GSO Opt@1Code88.279.4−8.8
OSWorld 2.1 full - reported binary completion, step limit unspecifiedAgents43.134.3−8.8
Roboflow Vision Evals (October 2026, five tasks)Multimodal74.783.4+8.7
SWE-TogetherCode69.360.6−8.7
Finance Agent v2 - Disclosure AnalysisFinance & Accounting70.361.7−8.6
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting77.068.4−8.5
Terminal-Bench 2.1Code81.689.9+8.3
Code2Video-AA EloCode13251427+8.3
Finance Agent v2 - ComparablesFinance & Accounting46.037.8−8.2
MindCube TinyMultimodal84.592.7+8.2
Harvey LAB (Mercor run, undisclosed ~213-task cohort)Legal17.49.2−8.2
Financial Coaching BenchFinance & Accounting75.767.5−8.2
Extended NYT Connections 940 (Quadratic v1)Games90.098.1+8.1
SnorkelFinance 2.0Finance & Accounting19.611.6−8.0
BullshitBench v2 LeaderboardSafety77.069.0−8.0
Snorkel Agentic Coding 2.0Code39.647.6+8.0
Build-A-Chip-BenchAgents16.68.7−7.9
GDPval-AA v2.1Professional Work17651587−7.8
Blueprint-Bench 2Multimodal0.40.5+7.8
LVBenchMultimodal79.787.5+7.8
Immigration BenchLegal75.467.7−7.8
CVDP (517-problem set)Code62.654.9−7.7
OpenAI internal computer-use safety - adverse outcome rateSafety9.51.8+7.7
AA-PersonalityBench Slop indexChat & Writing23.816.2−7.7
Arena Alignment IndexSafety80.187.8+7.6
Finance Agent v2 - All PassFinance & Accounting47.840.3−7.5
AA-LCR v1.1Search87.580.0−7.5
ArXivMath 05/2026Math87.595.0+7.5
Gray Swan IPI Attack Success (k=15)Safety1.08.5−7.5
HardGeoBench <=10miMultimodal45.037.5−7.5
FrontierSWE V2 (Mercor rerun, mean@3)Code37.229.9−7.3
GraphWalks BFS: Up to 128KGeneral QA91.498.7+7.3
τ³-Banking (Mercor run, bm25_grep retrieval)Agents54.046.7−7.3
TextQuests Average Cumulative HarmSafety27.129.2−7.1
JobBench - release-reported mean task rewardProfessional Work73.166.1−7.0
SWE-Clar (implementation)Code60.067.0+7.0
HealthBenchHealth & Medicine63.956.9−7.0
Toloka Arena (v1)Agents60.753.8−6.9
Qodo AI Code Review Benchmark (125 PRs)Code53.546.6−6.9
DeepSWE v1.1Code67.374.1+6.8
GraphWalks BFS: 256K to 1M AggregateGeneral QA65.071.8+6.8
GDP.pdfProfessional Work27.634.2+6.6
SciCode 1.0.1 (Artificial Analysis)Code63.156.5−6.6
RiemannBenchMath65.672.0+6.4
FoodTruck BenchGames6273257761−6.4
AA-AnalystAgent (pass^5)Professional Work57.551.3−6.2
Humanity's Last ExamGeneral QA60.954.8−6.1
Finance Agent v2 - AdjustmentsFinance & Accounting54.448.3−6.1
CritPt v2 (agentic, avg@5)Science77.783.7+6.0
Finance Agent v2 - Earnings AnalysisFinance & Accounting75.569.5−6.0
AA-Briefcase EloAgents16621562−5.8
SUPERChemScience74.780.3+5.6
FrontierCUA pass@1Agents36.742.2+5.6
OpenDS-WorldSearch33.539.0+5.5
MysteryMechanismScience47.753.2+5.4
DAXBench v2Code91.596.8+5.3
Roboflow Vision Evals (September 2026 protocol)Multimodal81.386.6+5.3
Search ArenaSearch12371231−5.2
MedXpertQA MM Extended (Mercor held-out)Health & Medicine66.060.8−5.2
MedCodeHealth & Medicine53.548.5−5.0
TasteVal (compute multiplier)Science1.71.6−5.0
TxBench-Antibody-Development V1 (60 evaluations)Science27.832.8+5.0
ARC-AGI-2General QA90.095.0+5.0
LMArena Text Arena EloChat & Writing15031474−5.0
Excel Modeling BenchmarkFinance & Accounting76.771.7−5.0
DeepResearch Bench II (Mercor 130-task set)Search55.050.1−4.9
OSWorld 2.1 full - reported partial reward, step limit unspecifiedAgents80.775.8−4.9
LMArena Agent Arena SteerabilityAgents8.4-1.4−4.9
DAYJOB: TaxFinance & Accounting9.64.8−4.8
GPQA DiamondGeneral QA91.496.2+4.8
SimpleQA Verified (Epoch anti-abstention)General QA70.875.6+4.8
DAYJOB: Data ScienceProfessional Work26.021.3−4.7
SWE Atlas v1.1 - OverallCode74.169.4−4.7
GDPval Extended (Mercor held-out rubric score)Professional Work68.764.1−4.6
SciCode (main problems)Science24.219.6−4.6
APEX-1 (Mercor, mean score)Professional Work73.869.3−4.5
SWE Atlas - Codebase QnACode43.347.8+4.5
AA-3D v1.0 EloCode11391168+4.3
MA-BenchProfessional Work49.253.4+4.2
OSWorld 2.1 offline - reported partial reward, step limit unspecifiedAgents76.880.9+4.1
PostTrainBench Average (v1.1)Indexes40.244.3+4.1
Excel Modeling Benchmark v1.1Finance & Accounting76.772.7−4.0
PCT Political Inconsistency (CAIS)Safety33.437.4−4.0
Toloka Arena (v3)Agents71.067.0−4.0
HackerRank-BenchCode47.543.5−4.0
Finance Agent v2 - Market AnalysisFinance & Accounting69.565.5−4.0
DualEntry Accounting AI BenchmarkFinance & Accounting85.181.2−3.9
Drone-BenchAgents91.295.1+3.9
AA-PersonalityBench Transparency indexChat & Writing98.194.3−3.8
HiL-Bench (Human-in-Loop Benchmark)Agents61.565.3+3.8
Westworld Finance Diligence BenchFinance & Accounting58.254.5−3.8
RT-SAFE Collisions per EpisodeSafety20.922.4−3.8
Finance Agent v2 - Financial ModelingFinance & Accounting25.321.6−3.7
LLM Creative Story WritingChat & Writing3.83.4−3.5
FrontierMath Tiers 1-3 (v2) - PrivateMath90.293.7+3.5
ArXivMath 06/2026Math91.094.4+3.5
MedScribeHealth & Medicine91.387.9−3.4
BenchCAD - geometric overlapMultimodal92.695.9+3.3
micro1 PharmaBenchHealth & Medicine12.816.1+3.3
ProgramBench (Mercor 100-task subset)Code11.38.0−3.3
Professional Reasoning Benchmark - FinanceFinance & Accounting50.847.5−3.3
Professional Reasoning Benchmark - LegalLegal51.648.4−3.2
SWE-Clar (clarification)Code84.387.5+3.2
APEX-AccountingFinance & Accounting61.057.9−3.1
SnorkelRevOpsProfessional Work14.010.9−3.1
MedXpertQA MM (200-question set)Health & Medicine83.786.8+3.1
LMArena Agent Arena Bash RecoveryAgents11.65.5−3.0
Shroombench (mushroom identification, top-1)Multimodal51.054.0+3.0
SimpleBenchGeneral QA86.683.6−3.0
FrontierMath ErdősMath0.02.9+2.9
Remote Labor Index (RLI)Professional Work17.920.8+2.9
BuseyBench SVGMultimodal7.07.3+2.9
CharXiv Extended (Mercor held-out)Multimodal82.285.1+2.9
SnorkelUnderwrite 2.0Finance & Accounting24.021.2−2.8
DAYJOB: AccountingFinance & Accounting27.230.0+2.8
Mercor enterprise risk (benign-control completion rate)Safety100.097.2−2.8
Mercor enterprise risk (implicit-risk unsafe rate)Safety77.880.6−2.8
DistressBenchSafety71.574.3+2.8
HardGeoBench <=100miMultimodal53.851.3−2.5
Dual-Arm Sim (task completion)Agents0.02.5+2.5
Longform Creative WritingChat & Writing85.382.8−2.5
LibraryDesignBenchCode47.545.1−2.4
Harvey LAB-AA v1.1 (hallucination-gated)Legal6.48.6+2.2
AdvancedIFChat & Writing84.282.0−2.2
SAGEMath48.546.4−2.2
Snorkel SWE-bench CLICode14.512.4−2.1
Senior SWE-BenchCode34.736.8+2.1
APEX-Agents 1.1Professional Work77.175.1−2.0
DAYJOB: HealthcareHealth & Medicine9.611.6+2.0
TaxCalcBench TY25Finance & Accounting42.044.0+2.0
ACCoRDGeneral QA83.781.7−2.0
HardGeoBench Avg DistanceMultimodal845908−1.9
Finance Agent v2 - PrecedentsFinance & Accounting32.230.3−1.9
NoCoT-Bench NCKI (kspine v3)General QA128131+1.9
LiveBench 2.0Agents88.086.1−1.8
Epoch Capabilities Index (ECI)Indexes167169+1.7
DAYJOB: FinanceFinance & Accounting19.821.5+1.7
SAB ResearchFinance & Accounting96.895.2−1.7
Creative Writing v3 Rubric ScoreChat & Writing16.315.9−1.7
FORTRESS Average Risk Score (ARS)Safety13.712.1+1.6
Vision Arena OverallMultimodal12891283−1.5
MissionReportProfessional Work63.361.8−1.5
MMLU ProGeneral QA93.091.5−1.5
Conceptual Reasoning IndexIndexes74.973.9−1.5
Snorkel WorkplaceAgentsProfessional Work15.814.4−1.4
SAB RankingForecasting-0.1-0.0+1.4
SnorkelManufacturingProfessional Work11.310.0−1.3
HardGeoBench Country AccuracyMultimodal58.860.0+1.3
LMArena Agent Arena Bash ReliabilityAgents10.17.8−1.1
sudo L7 (October 2026 rescoring)Code32.231.1−1.1
Creative Writing v3 EloChat & Writing21552177+1.1
LMCAGeneral QA65.564.4−1.1
GDPxProfessional Work8.59.6+1.1
CloudOpsBench v1.0Agents43.842.8−1.0
Ramp Accounting BenchFinance & Accounting49.748.7−1.0
HealthBench (length-adjusted, Artificial Analysis)Health & Medicine57.258.2+1.0
ARC-AGI-1 Semi-PrivateGeneral QA97.598.5+1.0
CharXiv (descriptive and reasoning combined)Multimodal95.196.1+1.0
ProofBench v1.1Math100.099.0−1.0
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting79.778.8−1.0
ITBench-AAAgents49.548.6−0.9
FrontierCode 1.1 Extended - ScoreCode63.664.5+0.9
Ruby LLM Benchmarks — Program FixerCode76.075.1−0.8
CheatBenchSafety45.845.0+0.8
ObviousBenchGeneral QA99.3100.0+0.7
ATLAS-Finance v1.0Finance & Accounting12.011.3−0.7
ATLAS-Finance v1.1Finance & Accounting10.39.7−0.7
Snorkel Digital Work IndexIndexes23.623.0−0.6
CritPt (avg@5)Science31.131.7+0.6
Artificial Analysis Intelligence Index v4.3Indexes53.452.8−0.6
LMArena Agent Arena Net Improvement (Steering Burden)Agents14.814.5−0.5
HealthBench ProfessionalHealth & Medicine69.269.7+0.5
WeirdML (v2)Science92.993.3+0.4
MMMU ProMultimodal86.186.4+0.3
Prophet Arena Arena Score (Fixed Context)Forecasting85.084.7−0.3
GameBench 2Games94.094.3+0.3
DTBenchGeneral QA97.697.3−0.3
AutoResearchExam (hidden-test AUARC)Agents0.60.6−0.2
GDP.xlsxProfessional Work23.122.9−0.2
LMArena Agent Arena Confirmed SuccessAgents14.414.0−0.2
Prophet Arena Arena Score (Agentic, OpenCode)Forecasting86.086.2+0.1
LMArena Agent Arena Tool HallucinationAgents0.81.0+0.1
GDPval (Mercor 213-task rubric score)Professional Work86.686.7+0.1
Terminal-Bench 4.0Code57.957.90.0
LMArena Agent Arena Praise vs ComplaintAgents31.631.70.0
LMArena Agent Arena Steering BurdenAgents14.814.80.0
AIME 2025Math100.0100.00.0
BabaIsBench Stage 0: The IntroGames100.0100.00.0
MBABench 2.0Finance & Accounting7.97.90.0
Mercor enterprise risk (explicit-risk unsafe rate)Safety0.00.00.0
Next.js AI Agent EvaluationsAgents100.0100.00.0
OTIS Mock AIME 2024-2025Math100.0100.00.0

Claude Fable 5.1vsGPT-6 Astranormalised score marginsData Nov 15, 2026Data: Mercor, Artificial Analysis, Vals AI and 81 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…