Atlas

Compare Models

Claude Fable 5.1 vs Gemini 3.8 Flash

Claude Fable 5.1 leads by 5.1 capability points (≈95% interval +2.7 to +7.4): a clear lead over Gemini 3.8 Flash · P(higher) ≈ 100%

Wins 203 of 262 shared benchmarks (56 lost, 3 tied)

Capability vs. relative cost
Claude Fable 5.1Gemini 3.8 FlashClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Fable 5.1vsGemini 3.8 Flashcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins203 – 56 · 262 shared, 3 tied

Comparing matching setups only (28 benchmarks excluded) · suites counted once
← Claude Fable 5.1Gemini 3.8 Flash →

Claude Fable 5.1vsGemini 3.8 Flashwins by categoryData Nov 15, 2026Data: Vals AI, Mercor, Artificial Analysis and 75 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Gemini 3.8 Flash minus Claude Fable 5.1.

VCT-Refusal (CAIS)Safety98.96.4−92.5
HPCT Refusal (CAIS)Safety100.013.0−87.0
AA-PersonalityBench Transparency indexChat & Writing98.123.8−75.0
FrontierMath Tier 4 (v2) - PrivateMath87.822.0−65.9
BrokenArXiv 06/2026Math86.620.8−65.7
BrokenArXiv 05/2026Math78.019.0−59.0
BrokenArXiv 08/2026 (Native Harnesses)Math79.827.4−52.4
ProofBench v1.1Math100.048.0−52.0
Medical Long Context Reasoning (MLCR-AA)Health & Medicine71.121.7−49.4
AA-PersonalityBench Slop indexChat & Writing23.871.4+48.1
ArXivMath 08/2026 (Native Harnesses)Math87.740.4−47.4
AA-PersonalityBench Imagination indexChat & Writing86.741.0−46.2
Terminal-Bench 4.0Code57.913.1−44.7
Time Horizon Index: KSPAgents63.318.8−44.5
BioSecBench-Refusal V1 Clean RerunsSafety5.647.4+41.8
LLM Creative Story WritingChat & Writing3.80.1−40.8
Furniture Assembly BenchmarkMultimodal70.031.7−38.3
Design Creativity BenchCode48.660.6+37.9
FrontierML ($200 wallet)Agents41.95.7−36.2
TaxCalcBench TY25Finance & Accounting42.06.0−36.0
AA-PersonalityBench Warmth indexChat & Writing94.360.0−34.6
Code2Video-AA EloCode1325899−34.6
HieroglyphBenchMultimodal20.454.6+34.2
LMArena Agent Arena Net ImprovementAgents14.13.8−34.0
ChessBench CoherenceGames61.994.6+32.7
LLM Debate BenchmarkChat & Writing17381514−31.8
SciUniverse Level 1Science45.314.6−30.7
GameBench 2Games94.064.0−30.1
Document ArenaProfessional Work15151482−29.4
ArXivMath 05/2026Math87.558.3−29.2
Arena Image-to-WebDevCode17241594−28.3
ChessBenchGames45.373.3+28.0
HANDBOOK.mdProfessional Work38.810.8−28.0
MASKSafety85.958.6−27.3
APEX-SWECode63.636.4−27.2
Terminal-Bench Science 0.1 (Vals)Science34.38.6−25.7
ChessBench EloGames10951527+25.7
CAD ArenaAgents66.241.0−25.1
Terminal-Bench 2.1 Extended (Mercor held-out)Code19.544.4+24.9
FrontierSWE V2 (Mercor rerun, mean@3)Code37.212.8−24.4
Concept Synth INDUCTION Challenge100 Round 1General QA33.09.0−24.0
AA-3D v1.0 EloCode1139974−24.0
Bug Hunt BenchCode41.017.1−23.8
ALE-BenchCode28492182−23.6
LMArena Agent Arena Net Improvement (Steering Burden)Agents14.82.3−23.6
ArXivMath 06/2026Math91.067.4−23.6
Biopharma Bench V0.1Health & Medicine69.946.6−23.3
Epoch Automation ReportsAgents65.342.3−23.1
Visa Agentic Commerce BenchmarkAgents75.653.3−22.2
Hedge-BenchFinance & Accounting26.84.7−22.1
AA-Briefcase EloAgents15781202−21.9
FrontierMath Tiers 1-3 (v2) - PrivateMath90.268.4−21.8
DAYJOB: AccountingFinance & Accounting27.26.0−21.2
LMArena Code Arena WebDevCode17421583−20.9
Creative Writing v3 EloChat & Writing21551744−20.8
LiveBench 2.0Agents88.067.3−20.6
Artificial Analysis Coding Agent Index v1.5Indexes62.241.9−20.4
Concept Synth INDUCTION Challenge64 Round 1General QA34.414.1−20.3
HiL-Bench (Human-in-Loop Benchmark)Agents61.541.5−20.0
CadQueryEval (Corrected Geometry)Code76.096.0+20.0
sudo L7Code31.111.7−19.4
AA-PersonalityBench Expressiveness indexChat & Writing98.179.1−19.2
EnterpriseBench: CoreCraftProfessional Work77.458.5−18.9
HoneyBench v0.2 (hack rate)Safety61.880.5−18.7
WeirdML v3Science26.07.4−18.6
Vals RSI Index v1.1 (historical five tasks)Agents35.017.6−17.5
AA-LCR Extended (Mercor held-out)Search73.055.6−17.4
RT-SAFE Collisions per EpisodeSafety20.927.7−17.0
Humanity's Last Exam DiamondGeneral QA51.334.3−17.0
DAYJOB: FinanceFinance & Accounting19.82.8−17.0
SWE Atlas v1.1 - OverallCode74.157.3−16.8
Humanity's Last ExamGeneral QA60.944.5−16.4
Olam Social Poker EloGames14941479−16.3
Vals RSI Index v1.1 (four tasks)Agents36.119.9−16.2
SWE-bench VerifiedCode96.680.6−16.0
Build-A-Chip-BenchAgents16.60.8−15.8
BioTIER RefusalSafety99.583.9−15.6
AA-PersonalityBench Informativeness indexChat & Writing96.281.0−15.4
AA-PersonalityBench Conviction indexChat & Writing10084.8−15.4
GeneBench-Pro Public Release SubsetScience26.011.0−15.0
RiemannBenchMath65.651.2−14.4
SnorkelFinance 2.0Finance & Accounting19.65.4−14.2
RSI-Exam 0.1 (Full 88)Agents0.50.3−14.1
Shroombench (mushroom identification, top-1)Multimodal51.065.0+14.0
BARC-Bench (all-safe rate)Safety37.323.6−13.6
SAGEMath48.535.1−13.5
AA-PersonalityBench Organization indexChat & Writing60.073.3+13.5
LMArena Agent Arena Praise vs ComplaintAgents31.64.9−13.4
ArXivMath 04/2026Math77.564.2−13.3
PCT Political Inconsistency (CAIS)Safety33.446.3−12.9
CritPt (avg@5)Science31.118.3−12.9
TxBench-Antibody-Development V1 (60 evaluations)Science27.815.0−12.8
Conceptual Reasoning IndexIndexes74.965.7−12.7
CritPt v2 (agentic, avg@5)Science77.765.1−12.6
LMCAGeneral QA65.552.9−12.6
Machiavelli Deception (CAIS)Safety82.486.5−12.4
Remote Labor Index (RLI)Professional Work17.95.8−12.1
Immigration BenchLegal75.463.4−12.0
Toloka Arena (v3)Agents71.059.0−12.0
CVDP (517-problem set)Code62.650.7−11.9
AutoResearchExam (hidden-test AUARC)Agents0.60.5−11.9
FrontierCode 2.0 (Artificial Analysis)Code30.318.7−11.6
FrontierPhysics v0.1Science11.30.0−11.3
MysteryMechanismScience47.736.5−11.3
AA-AnalystAgent (pass^5)Professional Work57.546.3−11.2
MMMU-Pro Extended (Mercor held-out)Multimodal47.636.4−11.2
Tax Agent BenchFinance & Accounting77.966.8−11.1
LegalReasonBenchLegal50.639.6−11.0
Mystery Game PuzzlesGames58.047.0−11.0
MentalHealthBenchSafety46.435.5−10.9
GBENCH One-shot Coding GScoreGames69.358.6−10.7
SnorkelLegalLegal35.925.3−10.6
ChessBench AccuracyGames71.381.1+10.5
ProgramBench (Mercor 100-task subset)Code11.31.0−10.3
HealthBench ProfessionalHealth & Medicine69.259.0−10.2
FrontierCode 1.1 Extended - ScoreCode63.653.5−10.2
SciCode (main problems)Science24.214.2−10.0
Chess Puzzles (task version 2.0)Games48.058.0+10.0
Financial Coaching BenchFinance & Accounting75.766.3−9.4
APEX-AccountingFinance & Accounting61.051.7−9.3
ERQAMultimodal70.379.6+9.3
TextQuests Average Cumulative HarmSafety27.124.4+9.2
FrontierCode 1.1 Extended - Pass RateCode68.859.6−9.2
GDPval-AA v2Professional Work16501464−9.1
MissionReportProfessional Work63.354.3−9.0
CUA-benchAgents13.24.2−9.0
Finance Agent v2 - Market AnalysisFinance & Accounting69.578.5+9.0
GDPval-AA v2.1Professional Work16441442−8.9
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting77.068.1−8.9
HealthBench (length-adjusted, Artificial Analysis)Health & Medicine57.248.4−8.8
DAYJOB: HealthcareHealth & Medicine9.60.8−8.8
DAYJOB: TaxFinance & Accounting9.60.8−8.8
SnorkelRevOpsProfessional Work14.05.4−8.6
SM BenchSafety89.080.4−8.6
Westworld Finance Diligence BenchFinance & Accounting58.249.7−8.6
Longform Creative WritingChat & Writing85.376.8−8.5
GDPval Extended (Mercor held-out rubric score)Professional Work68.760.4−8.3
Snorkel SWE-bench CLICode14.56.3−8.2
WeirdML (v2)Science92.984.8−8.1
LMArena Agent Arena Steering BurdenAgents14.8-1.3−8.0
Antidote: Everyday EditionSafety10651087+7.9
MindCube TinyMultimodal84.592.2+7.7
HardGeoBench <=100miMultimodal52.560.0+7.5
Extended NYT Connections 940 (Quadratic v1)Games90.097.4+7.4
LVBenchMultimodal79.787.1+7.4
MineBench — Bradley-Terry RatingGames19281806−7.4
ATLAS-Finance v1.0Finance & Accounting12.04.7−7.3
Finance Agent v2 - Financial ModelingFinance & Accounting25.332.6+7.3
AutomationBench 1.0.6 (Zapier)Agents22.429.7+7.3
SWE-bench Verified Extended (Mercor held-out)Code61.354.0−7.3
micro1 PharmaBenchHealth & Medicine12.85.6−7.2
EnigmaEvalGeneral QA44.237.2−7.0
Snorkel Agentic Coding 2.0Code39.632.6−7.0
Artificial Analysis Intelligence IndexIndexes65.758.7−7.0
Last Translation Benchmark (LTBv1-eval, blind)Chat & Writing32.939.8+6.9
MedScribeHealth & Medicine91.384.5−6.8
Epoch Capabilities Index (ECI)Indexes167159−6.8
HealthBenchHealth & Medicine63.957.1−6.7
Snorkel Digital Work IndexIndexes23.616.9−6.7
ATLAS-Finance v1.1Finance & Accounting10.33.7−6.7
LMArena Agent Arena Confirmed SuccessAgents14.41.3−6.6
Qodo AI Code Review Benchmark (125 PRs)Code53.547.0−6.5
SciCode 1.0.1 (Artificial Analysis)Code63.156.6−6.5
Harvey LAB-AA v1.1 (hallucination-gated)Legal6.40.0−6.4
HardGeoBench <=10miMultimodal40.046.3+6.3
Real-SWECode45.038.8−6.2
HardGeoBench Country AccuracyMultimodal61.367.5+6.2
LMArena Agent Arena Bash RecoveryAgents11.6-0.7−6.1
Terminal-Bench 2.1Code81.687.6+6.0
Finance Agent v2 - ComparablesFinance & Accounting46.052.0+6.0
Eval Connections One-Shot with TrapsGames82.088.0+6.0
Harvey LAB (Mercor run, undisclosed ~213-task cohort)Legal17.411.4−6.0
Finance Agent v2 - Earnings AnalysisFinance & Accounting75.581.4+5.9
FrontierFinance v3.1Finance & Accounting55.950.0−5.9
MA-BenchProfessional Work49.243.6−5.6
ACCoRDGeneral QA83.778.2−5.5
MortgageTax (Vals all 1,258 documents)Finance & Accounting70.865.3−5.4
MedCodeHealth & Medicine53.548.1−5.4
NoCoT-Bench NCKI (kspine v3)General QA128121−5.3
Design Arena Overall Code CategoriesCode13311299−5.3
ChartographyMultimodal46.240.9−5.3
SWE-bench MultilingualCode91.185.8−5.3
IntPhys 2Multimodal67.762.5−5.2
MedXpertQA MM Extended (Mercor held-out)Health & Medicine66.060.8−5.2
τ³-Banking (Mercor run, bm25_grep retrieval)Agents54.048.8−5.2
Roboflow Vision Evals (October 2026, five tasks)Multimodal74.779.8+5.1
SWE-TogetherCode69.364.2−5.1
Clipping Bench v1.0Games1068596−5.1
APEX-1 (Mercor, mean score)Professional Work73.868.8−5.0
SnorkelUnderwrite 2.0Finance & Accounting24.019.1−4.9
GDP.xlsxProfessional Work23.118.3−4.8
BrowseComp (Mercor 130-question subset)Search78.383.1+4.8
Snorkel WorkplaceAgentsProfessional Work15.811.2−4.6
Gray Swan IPI Attack Success (k=15)Safety1.05.5−4.5
Excel Modeling BenchmarkFinance & Accounting76.772.2−4.5
Agent Red Teaming k=100 (CAIS)Safety34.229.8+4.4
Arena Alignment IndexSafety80.175.8−4.3
DAXBench v2Code91.587.2−4.3
SnorkelManufacturingProfessional Work11.37.1−4.2
SimpleBenchGeneral QA86.682.4−4.2
BuseyBench SVGMultimodal7.06.6−4.0
WellnessBenchSafety51.047.0−4.0
Ramp Accounting BenchFinance & Accounting49.745.7−3.9
Finance Agent v2 - PrecedentsFinance & Accounting32.236.1+3.9
Excel Modeling Benchmark v1.1Finance & Accounting76.772.9−3.9
Roboflow Vision Evals (September 2026 protocol)Multimodal81.385.1+3.8
DeepSWE v1.1Code67.371.0+3.7
Realm Financial ReasoningFinance & Accounting47.851.5+3.7
BioSecBench-RespiratoryVirus-Surveillance V1 (110 evaluations)Science3.67.3+3.6
AA-LCR v1.1Search87.584.0−3.5
Creative Writing v3 Rubric ScoreChat & Writing16.315.6−3.3
AIME 2025Math100.096.7−3.3
Blueprint-Bench 2Multimodal0.40.4−3.3
CharXiv Extended (Mercor held-out)Multimodal82.278.9−3.3
ComplexConstraintsChat & Writing45.148.4+3.3
GDPval (Mercor 213-task rubric score)Professional Work86.683.5−3.1
ITBench-AAAgents49.552.5+3.0
OpenDS-WorldSearch33.536.5+3.0
Professional Reasoning Benchmark - LegalLegal51.648.9−2.7
SWE Atlas - Codebase QnACode43.340.6−2.7
SpeciEval (Refusal-Aware Retries, Historical)Safety87.289.9+2.6
SpeciEval (Pooled English Runs)Safety87.289.9+2.6
TextQuests Game ProgressGames56.654.1−2.5
MMLU ProGeneral QA93.090.7−2.3
SUPERChemScience74.772.6−2.1
LMArena Agent Arena SteerabilityAgents8.44.3−2.1
Finance Agent v2 - All PassFinance & Accounting47.849.7+1.9
DualEntry Accounting AI BenchmarkFinance & Accounting85.183.2−1.9
DTBenchGeneral QA97.695.7−1.9
Finance Agent v2 - Disclosure AnalysisFinance & Accounting70.368.5−1.8
MMMU ProMultimodal86.184.5−1.6
LegalBenchLegal88.587.0−1.5
TaxEval v2Finance & Accounting76.074.4−1.5
HardGeoBench Avg DistanceMultimodal888843+1.4
Professional Reasoning Benchmark - FinanceFinance & Accounting50.849.5−1.3
NoCoT-Bench NCRI 15.2General QA128126−1.2
OTIS Mock AIME 2024-2025Math100.098.9−1.1
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting79.780.9+1.1
APEX-Agents 1.1Professional Work71.872.9+1.1
SimpleQA Verified (Epoch anti-abstention)General QA70.869.7−1.1
CloudOpsBench v1.0Agents43.842.8−1.0
LiveCodeBench (Vals AI public dataset)Code90.589.5−1.0
GPQA DiamondGeneral QA93.994.9+1.0
ARC-AGI-1 Semi-PrivateGeneral QA97.598.5+1.0
MedXpertQA MM (200-question set)Health & Medicine83.784.7+1.0
Vision Arena OverallMultimodal12891293+1.0
LMArena Text Arena EloChat & Writing15031497−0.9
ARC-AGI-2General QA90.089.2−0.8
DistressBenchSafety71.570.7−0.8
ObviousBenchGeneral QA99.3100.0+0.7
RuneBenchGames6.06.0−0.6
Hemingway-benchChat & Writing10951093−0.6
Finance Agent v2 - AdjustmentsFinance & Accounting54.454.8+0.4
CharXiv (descriptive and reasoning combined)Multimodal95.195.4+0.3
Prophet Arena Arena Score (Fixed Context)Forecasting85.085.3+0.2
SpatialViz-BenchMultimodal77.978.1+0.2
LMArena Agent Arena Tool HallucinationAgents0.80.4−0.2
LMArena Agent Arena Bash ReliabilityAgents10.19.9−0.1
Prophet Arena Arena Score (Agentic, OpenCode)Forecasting86.086.00.0
BabaIsBench Stage 0: The IntroGames100.0100.00.0
Dual-Arm Sim (task completion)Agents0.00.00.0
FrontierMath ErdősMath0.00.00.0

Claude Fable 5.1vsGemini 3.8 Flashnormalised score marginsData Nov 15, 2026Data: Vals AI, Mercor, Artificial Analysis and 75 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…