Atlas

Compare Models

Claude Haiku 5.5 vs DeepSeek V4.1 Flash

Claude Haiku 5.5 leads by 1.9 capability points (≈95% interval −0.1 to +3.9): statistically tied · P(higher) ≈ 97%

Wins 77 of 143 shared benchmarks (64 lost, 2 tied)

Capability vs. relative cost
Claude Haiku 5.5DeepSeek V4.1 FlashClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Haiku 5.5vsDeepSeek V4.1 Flashcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins77 – 64 · 143 shared, 2 tied

Comparing matching setups only (23 benchmarks excluded) · suites counted once
← Claude Haiku 5.5DeepSeek V4.1 Flash →

Claude Haiku 5.5vsDeepSeek V4.1 Flashwins by categoryData Nov 15, 2026Data: Artificial Analysis, Vals AI, Mercor and 31 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is DeepSeek V4.1 Flash minus Claude Haiku 5.5.

AA-PersonalityBench Organization indexChat & Writing41.0100+59.6
KernelBench Mega Normalized (RTX PRO 6000)Code0.048.2+48.2
Medical Long Context Reasoning (MLCR-AA)Health & Medicine63.322.8−40.6
ProofBench v1.1Math89.054.0−35.0
AutomationBench-AA 1.0.6Agents36.068.9+32.9
AA-PersonalityBench Imagination indexChat & Writing63.831.5−32.7
SWE-bench Verified Extended (Mercor held-out)Code53.382.7+29.4
AA-PersonalityBench Conviction indexChat & Writing67.696.2+28.8
AA-PersonalityBench Warmth indexChat & Writing69.541.0−28.8
AA-PersonalityBench Slop indexChat & Writing10.537.2+26.9
Extended NYT Connections 940 (Quadratic v1)Games65.789.6+23.9
BrowseComp (Mercor 130-question subset)Search56.379.4+23.1
FrontierMath Tier 4 (v2) - PrivateMath46.326.8−19.5
SimpleQA Verified (Epoch anti-abstention)General QA23.842.9+19.1
DeepResearch Bench II (Mercor 130-task set)Search34.353.2+18.9
AA-AnalystAgent (pass^5)Professional Work53.835.0−18.7
APEX-Agents 1.1Professional Work68.350.5−17.8
AA-PersonalityBench Transparency indexChat & Writing77.260.0−17.3
AA-PersonalityBench Expressiveness indexChat & Writing77.261.9−15.4
MineBench — Bradley-Terry RatingGames15821822+14.5
CharXiv Extended (Mercor held-out)Multimodal82.968.9−14.0
MysteryMechanismScience35.121.2−14.0
ACCoRDGeneral QA81.568.1−13.5
Furniture Assembly BenchmarkMultimodal47.534.2−13.3
SWE-bench MultilingualCode85.198.2+13.1
A-Fantasia (Valid Attempts with Retries)Games72.359.3+13.0
Mystery Game PuzzlesGames30.043.0+13.0
DeepSWE v1.1Code59.972.6+12.7
LMArena Agent Arena Chat Net Improvement (Steering Burden)Agents-3.31.6+12.1
APEX-Accounting 1.1Finance & Accounting62.250.1−12.1
HealthBench ProfessionalHealth & Medicine64.852.9−11.8
ARC-AGI-2General QA61.372.9+11.7
Code2Video-AA EloCode11921053−11.3
Terminal-Bench-Science 0.1Science20.09.0−11.0
Humanity's Last Exam DiamondGeneral QA29.619.1−10.5
SpeciEval (Pooled English Runs)Safety86.997.4+10.4
MedCodeHealth & Medicine51.541.2−10.4
Terminal-Bench Science 0.1 (Vals)Science10.00.0−10.0
Conceptual Reasoning IndexIndexes65.358.4−9.4
LMArena Text Arena EloChat & Writing14221475+9.2
Vals Science IndexIndexes38.529.6−8.9
LMArena Agent Arena Chat Praise vs ComplaintAgents-14.33.1+8.7
AA-Briefcase v1.1 EloAgents15731415−8.6
CritPt (avg@5)Science22.614.3−8.3
DistressBenchSafety71.779.8+8.1
DualEntry Accounting AI BenchmarkFinance & Accounting76.284.2+8.0
GDP.pdf (Artificial Analysis, All-pass)Professional Work20.812.8−8.0
AA-Omniscience IndexIndexes10.7-5.3−8.0
FrontierMath Tiers 1-3 (v2) - PrivateMath75.167.4−7.7
GBENCH One-shot Coding GScoreGames55.648.0−7.7
Chess Puzzles (task version 2.0)Games55.048.0−7.0
IOI (Vals v2)Code47.340.3−7.0
AA-LCR Extended (Mercor held-out)Search63.870.8+7.0
MMMU-Pro Extended (Mercor held-out)Multimodal31.925.0−6.9
Wiz Cyber Model Arena v3.9Cybersecurity57.050.2−6.8
GameBench 2Games75.168.4−6.6
Code Migration v1.1Code52.458.9+6.6
DAXBench v2Code81.975.5−6.4
Terminal-Bench 4.0 (Artificial Analysis)Code32.826.8−6.1
Eval Connections One-Shot with TrapsGames67.073.0+6.0
SWE-bench VerifiedCode88.982.9−6.0
Design Arena Overall Code CategoriesCode12801316+6.0
ALE-BenchCode19961829−5.9
WeirdML v3Science11.75.9−5.7
Vibe Code Bench v1.1Code90.484.7−5.7
Terminal-Bench 2.1Code85.090.6+5.6
KernelBench CUDA Normalized (RTX PRO 6000)Code41.447.0+5.6
ARC-AGI-1 Semi-PrivateGeneral QA89.094.5+5.5
Harvey's Legal Agent BenchmarkLegal1.36.7+5.4
SUPERChemScience62.056.7−5.3
τ³-Banking (Mercor run, bm25_grep retrieval)Agents43.037.8−5.2
DTBenchGeneral QA95.089.9−5.1
SRE Bench — Fully SolvedCode5.70.8−5.0
SWE Atlas - Codebase QnACode48.753.5+4.8
BuseyBench SVGMultimodal4.64.1−4.7
LMArena Agent Arena Code Steering BurdenAgents11.22.3−4.5
SAGE v1.1Math89.885.4−4.4
LMArena Agent Arena Chat Bash ReliabilityAgents7.8-1.0−4.4
MedXpertQA MM (200-question set)Health & Medicine67.863.5−4.3
LMCAGeneral QA51.147.0−4.1
MedScribeHealth & Medicine89.585.5−4.0
CharXiv (descriptive and reasoning combined)Multimodal94.190.1−4.0
LMArena Agent Arena Code Net Improvement (Steering Burden)Agents4.52.7−4.0
Artificial Analysis Intelligence Index v4.3.2Indexes43.439.5−3.9
Immigration BenchLegal61.865.6+3.8
AA-PersonalityBench Informativeness indexChat & Writing75.379.1+3.8
LMArena Agent Arena Work Bash ReliabilityAgents7.50.1−3.7
LMArena Agent Arena Chat Confirmed SuccessAgents-2.34.9+3.6
LMArena Agent Arena Code Bash ReliabilityAgents6.9-0.2−3.6
LMArena Code Arena WebDevCode15941621+3.5
AdvancedIFChat & Writing80.884.0+3.2
APEX-SWECode55.151.9−3.2
SciCode 1.0.1 (Artificial Analysis)Code55.051.9−3.1
Epoch Capabilities Index (ECI)Indexes160156−3.1
LMArena Agent Arena Work Praise vs ComplaintAgents-2.53.7+3.1
LMArena Agent Arena Work Steering BurdenAgents7.11.5−2.8
FrontierCode 1.1 Extended - Pass RateCode64.762.0−2.7
BioMysteryBench (Vals Terminus 2)Science70.467.8−2.6
SWE-bench Multimodal (Vals v1, 480 Tasks)Code44.046.5+2.5
HealthBench (length-adjusted, Artificial Analysis)Health & Medicine58.560.9+2.4
Legal Research BenchLegal43.841.3−2.4
LiveBench 2.0Agents75.678.0+2.4
LLM Chess EloGames11431087−2.3
Excel Modeling Benchmark v1.1Finance & Accounting65.567.8+2.3
Terminal-Bench 4.0Code33.331.2−2.1
SAGEMath45.847.9+2.1
Artificial Analysis Progress Index v1Indexes326320−2.1
ProgramBenchCode2.50.5−2.0
TaxCalcBench TY25Finance & Accounting2.04.0+2.0
FrontierCode 1.1 Extended - ScoreCode58.456.6−1.8
Excel Modeling BenchmarkFinance & Accounting59.057.2−1.8
CyberBench v1.1 OverallCybersecurity75.573.7−1.8
LMArena Agent Arena Work Confirmed SuccessAgents2.25.7+1.7
AIME 2025Math100.098.3−1.7
Ramp Accounting BenchFinance & Accounting42.340.8−1.5
GDPval-AA v2.1Professional Work16291600−1.3
GDPval (Mercor 213-task rubric score)Professional Work85.586.7+1.2
LMArena Agent Arena Code Confirmed SuccessAgents4.26.6+1.2
Harvey LAB-AA v1.1 (hallucination-gated)Legal0.61.7+1.1
Public Benefits Bench v1.1Professional Work63.264.3+1.1
Code MigrationCode44.645.6+1.1
LMArena Agent Arena Work Net Improvement (Steering Burden)Agents2.12.5+1.0
LMArena Agent Arena Chat Steering BurdenAgents-2.2-0.5+0.9
Political Science LLM BenchmarkProfessional Work65.766.5+0.8
APEX-1 (Mercor, mean score)Professional Work64.765.5+0.8
MMLU ProGeneral QA87.488.2+0.8
LMArena Agent Arena Work Tool HallucinationAgents-1.00.5+0.8
Terminal-Bench 2.1 Extended (Mercor held-out)Code37.738.4+0.7
HealthBenchHealth & Medicine61.261.9+0.7
Finance Agent v2Finance & Accounting54.153.5−0.7
RCM-Bench-1Health & Medicine88.489.0+0.6
LMArena Agent Arena Code Praise vs ComplaintAgents1.93.0+0.6
OTIS Mock AIME 2024-2025Math98.998.3−0.6
LMArena Agent Arena Code Tool HallucinationAgents-0.60.5+0.5
Vals Finance IndexIndexes57.457.1−0.4
Tax Agent BenchFinance & Accounting62.262.5+0.3
LMArena Agent Arena Chat Tool HallucinationAgents-0.20.3+0.3
Bug Hunt BenchCode20.520.6+0.2
ARC-AGI-3 (Semi-Private)General QA2.82.7−0.1
Finance Agent v2.1Finance & Accounting65.465.4+0.1
Decision-Bench v1.1Professional Work25.325.30.0
BullshitBench v2 LeaderboardSafety57.057.00.0
MedXpertQA MM Extended (Mercor held-out)Health & Medicine48.448.40.0

Claude Haiku 5.5vsDeepSeek V4.1 Flashnormalised score marginsData Nov 15, 2026Data: Artificial Analysis, Vals AI, Mercor and 31 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…