Atlas

Compare Models

DeepSeek V4.1 Flash 1017 vs Kimi K3.1 Pro

Kimi K3.1 Pro leads by 2.9 capability points (≈95% interval +0.5 to +5.3): a clear lead over DeepSeek V4.1 Flash 1017 · P(higher) ≈ 99%

Wins 74 of 105 shared benchmarks (31 lost)

Capability vs. relative cost
DeepSeek V4.1 Flash 1017Kimi K3.1 ProClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

DeepSeek V4.1 Flash 1017vsKimi K3.1 Procapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins31 – 74 · 105 shared

Comparing matching setups only (65 benchmarks excluded) · suites counted once
← DeepSeek V4.1 Flash 1017Kimi K3.1 Pro →

DeepSeek V4.1 Flash 1017vsKimi K3.1 Prowins by categoryData Nov 15, 2026Data: Vals AI, SecureBio, Moonshot AI and 30 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Kimi K3.1 Pro minus DeepSeek V4.1 Flash 1017.

Terminal-Bench 4.0Code31.863.6+31.8
BuseyBench SVGMultimodal4.36.9+25.9
RiemannBench (November 2026 rescoring)Math38.460.8+22.4
ArXivMath 08/2026 (Native Harnesses)Math43.964.9+21.0
PerceptionBenchMultimodal47.768.3+20.6
SimpleQA Verified (Epoch anti-abstention)General QA42.562.2+19.7
FrontierMath Tier 4 (v2) - PrivateMath46.365.9+19.5
ArXivMath 09/2026 (Native Harnesses)Math45.364.2+18.9
Humanity's Last Exam DiamondGeneral QA20.839.4+18.6
τ³-Banking (Mercor run, bm25_grep retrieval)Agents41.659.8+18.2
Legal Research Bench ImmigrationLegal36.418.2−18.2
ZeroBench Main Mean pass@1 (five samples)Multimodal11.829.8+18.0
DAYJOB: LawLegal6.223.9+17.7
BrokenArXiv 08/2026 (Native Harnesses)Math46.764.3+17.6
τ³-Banking (Fireworks, banking_knowledge)Agents46.863.3+16.5
Legal Research Bench Business/CommercialLegal44.227.9−16.3
ZeroBench (Main questions, pass@5)Multimodal23.039.0+16.0
Legal Research Bench Weighted PassLegal86.371.4−14.8
AA-Briefcase v1.1 EloAgents14161678+14.4
GameBench 2Games69.984.0+14.1
Concept Synth INDUCTION Challenge64 Round 1General QA32.846.9+14.1
BrokenArXiv 09/2026 (Native Harnesses)Math49.763.5+13.8
Legal Research Bench FamilyLegal27.313.6−13.6
Concept Synth INDUCTION Challenge100 Round 1General QA21.034.0+13.0
CWE-bench v2 Pass@1Cybersecurity57.645.3−12.3
VoxelBench Text RatingGames20472216+12.0
CharXiv ReasoningMultimodal77.089.0+12.0
ZeroBench (Main questions, pass@5, with tools)Multimodal51.063.0+12.0
Artificial Analysis Coding Agent Index v2.1Indexes36.848.6+11.8
NoCoT-Bench NCRI 15.2General QA81.6103+11.7
TCSExamMath46.057.4+11.4
Tax Agent BenchFinance & Accounting74.963.9−11.0
Design Arena Overall Code CategoriesCode12891353+10.6
Legal Research Bench Constitutional/Civil RightsLegal42.131.6−10.5
NoCoT-Bench NCKI (kspine v3)General QA102116+10.2
CyberOffenseCybersecurity72.863.2−9.6
Bio Capabilities Index (SecureBio)Indexes149161+9.0
Code Migration v1.1 (Vals Index v2.2 subset)Code68.559.7−8.8
DAXBench v2Code77.786.2+8.5
Humanity's Last ExamGeneral QA42.651.1+8.5
BrokenArXiv 10/2026 (Native Harnesses)Math49.758.0+8.3
VCT v2 (SecureBio)Science38.546.2+7.7
EnterpriseBench: CoreCraftProfessional Work58.566.1+7.6
DeepsecBench — Revalidated V1 (233 issues)Code30.037.5+7.5
ArXivMath 10/2026 (Native Harnesses)Math38.345.7+7.4
Legal Research Bench Civil LitigationLegal37.029.6−7.4
Agents' Last Exam v1.1 (ALE v1.1)Agents36.143.4+7.4
Chartography (with tools)Multimodal83.090.2+7.2
Legal Research Bench CriminalLegal35.128.1−7.0
FrontierMath Tiers 1-3 (v2) - PrivateMath71.278.2+7.0
Mystery Game PuzzlesGames49.042.0−7.0
VulnPR-100Code54.047.0−7.0
MBCT (SecureBio)Science53.960.7+6.8
Android Bench 2.0Code17.324.0+6.7
Legacy-Bench (Fireworks, avg@3)Code56.363.0+6.7
Legal Research Bench HealthLegal66.773.3+6.7
VCT (SecureBio)Science42.949.4+6.5
FrontierML ($200 wallet)Agents23.529.6+6.1
DAYJOB: AccountingFinance & Accounting7.613.6+6.0
GPQA-Bio (SecureBio)Science87.882.2−5.6
LMArena Agent Arena Net Improvement (Steering Burden)Agents2.04.9+5.5
Harvey LAB (Mercor run, undisclosed ~213-task cohort)Legal18.813.8−5.0
Vals Finance IndexIndexes60.956.0−4.9
AutomationBench-AA 1.0.6Agents68.573.3+4.8
HoneyBench v0.2 (hack rate)Safety76.471.8+4.6
DAYJOB: Finance (October 2026 rescoring)Finance & Accounting12.517.0+4.5
Visa Agentic Commerce BenchmarkAgents57.853.3−4.4
GBENCH One-shot Coding GScoreGames53.858.2+4.4
MMMU ProMultimodal79.784.1+4.4
ALE-BenchCode20122132+4.2
LAB-Bench LitQA2 (SecureBio)Science49.053.1+4.1
DualEntry Accounting AI BenchmarkFinance & Accounting82.286.1+3.9
LAB-Bench ProtocolQA (SecureBio)Science81.978.1−3.8
Access to Justice BenchmarkLegal73.169.5−3.6
APEX-Agents 1.1Professional Work79.475.9−3.5
LMArena Agent Arena Praise vs ComplaintAgents1.28.0+3.4
LAB-Bench CloningScenarios (SecureBio)Science94.891.5−3.3
Epiq LAER CorporateBenchProfessional Work75.278.4+3.2
LMArena Code Arena WebDevCode16641688+3.1
APEX-Accounting 1.1Finance & Accounting59.362.4+3.1
Legal Research Bench Administrative/RegulatoryLegal59.156.1−3.0
HPCT (SecureBio)Science60.563.5+3.0
Bug Hunt BenchCode37.334.3−3.0
BabyVision with PythonMultimodal92.595.4+2.9
Vibe Code Bench v1.1Code87.784.9−2.8
HLAB-Vals-1.1Legal14.211.7−2.5
LiveBench 2.0Agents79.276.7−2.5
LMArena Agent Arena Bash ReliabilityAgents2.47.2+2.4
Political Science LLM BenchmarkProfessional Work66.268.6+2.4
Epoch Capabilities Index (ECI)Indexes159162+2.2
WCB (SecureBio)Science49.651.5+1.9
WMDP-Bio (SecureBio)Science87.989.7+1.8
ErdosBenchMath3.03.0−1.7
Finance Agent v2.1Finance & Accounting62.961.5−1.4
DAYJOB: Data ScienceProfessional Work3.34.7+1.4
LMArena Agent Arena Steering BurdenAgents4.06.7+1.3
PubMedQA (SecureBio)Health & Medicine80.081.2+1.2
FrontierCode 2.0 - Pass RateCode23.022.0−1.0
Chess Puzzles (task version 2.0)Games46.045.0−1.0
Excel Modeling Benchmark v1.1Finance & Accounting69.270.0+0.8
OTIS Mock AIME 2024-2025Math99.4100.0+0.6
MMLU-Bio (SecureBio)Science90.791.2+0.5
LMArena Agent Arena Tool HallucinationAgents0.31.0+0.4
FrontierCode 2.0 - ScoreCode20.219.9−0.3
LMArena Agent Arena Confirmed SuccessAgents2.32.7+0.2

DeepSeek V4.1 Flash 1017vsKimi K3.1 Pronormalised score marginsData Nov 15, 2026Data: Vals AI, SecureBio, Moonshot AI and 30 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…