Atlas

Compare Models

Claude Fable 5.1 vs Gemini 3.7 Flash

Claude Fable 5.1 leads by 6.3 capability points (≈95% interval +4.1 to +8.6): a clear lead over Gemini 3.7 Flash · P(higher) ≈ 100%

Wins 139 of 181 shared benchmarks (41 lost, 1 tied)

Capability vs. relative cost
Claude Fable 5.1Gemini 3.7 FlashClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Fable 5.1vsGemini 3.7 Flashcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins139 – 41 · 181 shared, 1 tied

Comparing matching setups only (11 benchmarks excluded) · suites counted once
← Claude Fable 5.1Gemini 3.7 Flash →

Claude Fable 5.1vsGemini 3.7 Flashwins by categoryData Nov 15, 2026Data: Mercor, Arena, Vals AI and 45 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Gemini 3.7 Flash minus Claude Fable 5.1.

VCT-Refusal (CAIS)Safety98.94.8−94.1
HPCT Refusal (CAIS)Safety100.013.0−87.0
BrokenArXiv 06/2026Math86.611.1−75.5
BrokenArXiv 05/2026Math78.08.5−69.5
Medical Long Context Reasoning (MLCR-AA)Health & Medicine71.115.0−56.1
Terminal-Bench 4.0Code57.96.1−51.8
FrontierMath Tier 4 (v2) - PrivateMath87.836.6−51.2
LMArena Agent Arena Net ImprovementAgents14.1-1.3−50.7
LLM Creative Story WritingChat & Writing3.8-0.7−50.4
BioSecBench-Refusal V1 Clean RerunsSafety5.654.8+49.2
Furniture Assembly BenchmarkMultimodal70.026.7−43.3
ProofBench v1.1Math100.058.0−42.0
LLM Debate BenchmarkChat & Writing17381455−40.2
ArXivMath 05/2026Math87.547.5−40.0
HieroglyphBenchMultimodal20.455.0+34.6
ALE-BenchCode28491876−34.5
Document ArenaProfessional Work15151476−34.3
MASKSafety85.953.7−32.2
MirrorCode (ML, +Private, 2L)Code73.341.1−32.2
LMArena Agent Arena Net Improvement (Steering Burden)Agents14.8-2.2−32.1
SWE-bench Verified Extended (Mercor held-out)Code61.330.0−31.3
Terminal-Bench Science 0.1 (Vals)Science34.35.7−28.6
ArXivMath 06/2026Math91.062.5−28.5
Olam Social Poker EloGames14941468−28.3
TaxCalcBench TY25Finance & Accounting42.014.0−28.0
HANDBOOK.mdProfessional Work38.810.8−28.0
Arena Code WebDev — ReactCode17881584−27.3
BabyVisionMultimodal46.473.5+27.1
AA-Briefcase EloAgents15781116−26.9
Arena Image-to-WebDevCode17241600−26.9
Arena Code WebDev — Brand & MarketingCode17371553−26.8
RiemannBenchMath65.639.2−26.4
Arena Code WebDev — Reference-Based DesignCode18001608−25.3
Terminal-Bench 2.1 Extended (Mercor held-out)Code19.544.4+24.9
Arena Code WebDev — GamingCode18591638−24.7
AA-LCR Extended (Mercor held-out)Search73.048.9−24.1
Arena Code WebDev — FrontendCode17761592−23.5
Arena Code WebDev — SimulationsCode18181618−22.6
Creative Writing v3 EloChat & Writing21551720−22.0
Concept Synth INDUCTION Challenge100 Round 1General QA33.011.0−22.0
APEX-SWECode63.642.0−21.6
EnterpriseBench: CoreCraftProfessional Work77.455.9−21.5
ChessBench CoherenceGames61.983.2+21.3
Mystery Game PuzzlesGames58.037.0−21.0
LiveBench 2.0Agents88.067.2−20.8
Arena Code WebDev — Content Creation ToolsCode17051574−20.7
Tax Agent BenchFinance & Accounting77.957.7−20.2
ArXivMath 04/2026Math77.557.5−20.0
Senior SWE-BenchCode34.714.7−20.0
Bug Hunt BenchCode41.021.0−20.0
HackerRank-BenchCode47.528.5−19.0
FrontierMath Tiers 1-3 (v2) - PrivateMath90.271.6−18.6
Chess Puzzles (task version 2.0)Games48.066.0+18.0
Vals RSI Index v1.1 (four tasks)Agents36.118.3−17.8
LMArena Agent Arena Praise vs ComplaintAgents31.6-2.9−17.3
RSI-Exam 0.1 (Full 88)Agents0.50.3−17.2
Concept Synth INDUCTION Challenge64 Round 1General QA34.417.2−17.2
DAYJOB: FinanceFinance & Accounting19.82.8−17.0
CritPt (avg@5)Science31.114.3−16.9
GBENCH One-shot Coding GScoreGames69.352.6−16.7
SWE-bench VerifiedCode96.681.0−15.6
Arena Code WebDev — FullstackCode16881629−15.4
BioTIER RefusalSafety99.584.4−15.1
LMCAGeneral QA65.550.4−15.1
Conceptual Reasoning IndexIndexes74.964.6−14.3
Artificial Analysis Intelligence Index v4.3Indexes53.439.4−13.9
LABBench2Science68.682.1+13.5
Toloka Arena (v3)Agents71.058.0−13.0
Remote Labor Index (RLI)Professional Work17.95.0−12.9
Arena Code WebDev — HTMLCode16811593−12.4
GeneBench-Pro Public Release SubsetScience26.014.0−12.0
PCT Political Inconsistency (CAIS)Safety33.445.1−11.7
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting77.065.3−11.7
DeepResearch Bench II (Mercor 130-task set)Search55.043.4−11.6
MMMU-Pro Extended (Mercor held-out)Multimodal47.636.4−11.2
LMArena Agent Arena Steering BurdenAgents14.8-7.3−11.1
Arena Code WebDev — Consumer ProductCode16611574−11.0
GDPval Extended (Mercor held-out rubric score)Professional Work68.757.8−10.9
GDPval-AA v2Professional Work16501433−10.6
GDPval-AA v2.1Professional Work16441404−10.6
EnigmaEvalGeneral QA44.233.9−10.3
ProgramBench (Mercor 100-task subset)Code11.31.0−10.3
ChessBench EloGames10951265+10.1
Shroombench (mushroom identification, top-1)Multimodal51.061.0+10.0
MineBench — Bradley-Terry RatingGames19281765−9.9
DAYJOB: HealthcareHealth & Medicine9.60.0−9.6
APEX-Accounting 1.1Finance & Accounting66.757.2−9.5
Machiavelli Deception (CAIS)Safety82.485.5−9.4
TextQuests Average Cumulative HarmSafety27.124.4+9.2
LMArena Agent Arena Confirmed SuccessAgents14.4-3.5−9.0
LMArena Agent Arena Bash RecoveryAgents11.6-5.4−8.5
SWE-bench MultilingualCode91.182.6−8.5
ChessBench AccuracyGames71.379.2+8.5
SM BenchSafety89.080.6−8.4
Gray Swan IPI Attack Success (k=15)Safety1.09.2−8.2
CadQueryEval (Corrected Geometry)Code76.084.0+8.0
APEX-Agents 1.1Professional Work71.879.4+7.6
ERQAMultimodal70.377.7+7.4
Ramp Accounting BenchFinance & Accounting49.742.3−7.4
MedScribeHealth & Medicine91.383.9−7.4
FrontierCode 1.1 Extended - ScoreCode63.656.3−7.3
ATLAS-Finance v1.1Finance & Accounting10.33.3−7.0
Epoch Capabilities Index (ECI)Indexes167158−7.0
APEX-1 (Mercor, mean score)Professional Work73.867.0−6.8
ACCoRDGeneral QA83.777.0−6.7
FrontierCode 1.1 Extended - Pass RateCode68.862.3−6.5
HardGeoBench <=10miMultimodal40.046.3+6.3
HardGeoBench Country AccuracyMultimodal61.367.5+6.2
Toloka Arena (v1)Agents60.754.6−6.1
ChartographyMultimodal46.240.4−5.8
Arena Alignment IndexSafety80.174.4−5.7
LVBenchMultimodal79.785.4+5.7
Arena Code WebDev — Data & AnalyticsCode16151577−5.7
ARC-AGI-2General QA90.084.6−5.4
Roboflow Vision Evals (October 2026, five tasks)Multimodal74.780.1+5.4
Excel Modeling BenchmarkFinance & Accounting76.771.3−5.3
Creative Writing v3 Rubric ScoreChat & Writing16.315.2−5.2
NoCoT-Bench NCKI (kspine v3)General QA128121−5.0
Finance Agent v2 - Market AnalysisFinance & Accounting69.574.5+5.0
AA-LCR v1.1Search87.583.0−4.5
Design Arena Overall Code CategoriesCode13311304−4.5
Finance Agent v2 - ComparablesFinance & Accounting46.050.3+4.3
Agent Red Teaming k=100 (CAIS)Safety34.238.5−4.2
LMArena Agent Arena SteerabilityAgents8.4-0.0−4.2
Terminal-Bench 2.1Code81.685.8+4.2
MortgageTax (Vals all 1,258 documents)Finance & Accounting70.866.7−4.1
BuseyBench SVGMultimodal7.06.6−3.9
MindCube TinyMultimodal84.588.4+3.9
Roboflow Vision Evals (September 2026 protocol)Multimodal81.385.2+3.9
HardGeoBench <=100miMultimodal52.556.3+3.8
TextQuests Game ProgressGames56.652.9−3.7
Finance Agent v2 - Earnings AnalysisFinance & Accounting75.579.1+3.6
Antidote: Everyday EditionSafety10651075+3.6
AIME 2025Math100.096.7−3.3
MedXpertQA MM Extended (Mercor held-out)Health & Medicine66.062.7−3.3
SciCode 1.0.1 (Artificial Analysis)Code63.159.8−3.2
CharXiv Extended (Mercor held-out)Multimodal82.279.1−3.1
OTIS Mock AIME 2024-2025Math100.097.2−2.8
Hemingway-benchChat & Writing10951105+2.8
ComplexConstraintsChat & Writing45.142.4−2.7
SpatialViz-BenchMultimodal77.975.3−2.6
DistressBenchSafety71.569.0−2.6
Finance Agent v2 - Disclosure AnalysisFinance & Accounting70.367.7−2.5
AA-AnalystAgent (pass^5)Professional Work57.560.0+2.5
MMLU ProGeneral QA93.090.6−2.4
LMArena Text Arena EloChat & Writing15031489−2.3
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting79.781.8+2.1
ARC-AGI-1 Semi-PrivateGeneral QA97.595.5−2.0
Eval Connections One-Shot with TrapsGames82.080.0−2.0
LMArena Agent Arena Bash ReliabilityAgents10.16.2−1.9
SUPERChemScience74.772.8−1.9
LiveCodeBench (Vals AI public dataset)Code90.588.7−1.9
SpeciEval (Refusal-Aware Retries, Historical)Safety87.289.0+1.8
SpeciEval (Pooled English Runs)Safety87.289.0+1.8
DeepSWE v1.1Code67.365.5−1.8
Finance Agent v2 - PrecedentsFinance & Accounting32.233.8+1.6
SimpleQA Verified (Epoch anti-abstention)General QA70.869.2−1.6
GPQA DiamondGeneral QA93.995.5+1.5
GDPval (Mercor 213-task rubric score)Professional Work86.685.2−1.4
LegalBenchLegal88.587.3−1.3
MMMU ProMultimodal86.184.9−1.2
TaxEval v2Finance & Accounting76.074.7−1.2
Finance Agent v2 - Financial ModelingFinance & Accounting25.324.2−1.2
Vision Arena OverallMultimodal12891294+1.1
NoCoT-Bench NCRI 15.2General QA128126−1.1
BrowseComp (Mercor 130-question subset)Search78.379.2+0.9
Humanity's Last ExamGeneral QA46.545.6−0.9
DTBenchGeneral QA97.696.8−0.8
SAGEMath48.549.2+0.7
ObviousBenchGeneral QA99.3100.0+0.7
RuneBenchGames6.06.0−0.6
IntPhys 2Multimodal67.768.2+0.5
Finance Agent v2 - AdjustmentsFinance & Accounting54.454.7+0.3
Finance Agent v2 - All PassFinance & Accounting47.847.5−0.3
HardGeoBench Avg DistanceMultimodal888895−0.2
CharXiv (descriptive and reasoning combined)Multimodal95.195.3+0.2
MedXpertQA MM (200-question set)Health & Medicine83.783.5−0.2
GlobeBenchGeneral QA21.621.8+0.1
MedCodeHealth & Medicine53.553.4−0.1
LMArena Agent Arena Tool HallucinationAgents0.80.7−0.1
BabaIsBench Stage 0: The IntroGames100.0100.00.0

Claude Fable 5.1vsGemini 3.7 Flashnormalised score marginsData Nov 15, 2026Data: Mercor, Arena, Vals AI and 45 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…