Atlas

Compare Models

GPT-6 Sol vs Kimi K3.1 Pro

Kimi K3.1 Pro leads by 0.1 capability points (≈95% interval −2.3 to +2.4): statistically tied · P(higher) ≈ 52%

GPT-6 Sol wins 46 of 86 shared benchmarks (38 lost, 2 tied)

Capability vs. relative cost
GPT-6 SolKimi K3.1 ProClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

GPT-6 SolvsKimi K3.1 Procapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins46 – 38 · 86 shared, 2 tied

Comparing matching setups only (60 benchmarks excluded) · suites counted once
← GPT-6 SolKimi K3.1 Pro →

GPT-6 SolvsKimi K3.1 Prowins by categoryData Nov 15, 2026Data: Vals AI, SecureBio, Epoch AI and 20 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Kimi K3.1 Pro minus GPT-6 Sol.

ArXivMath 08/2026 (Native Harnesses)Math93.064.9−28.1
FrontierMath Tier 4 (v2) - PrivateMath90.065.9−24.1
EnterpriseBench: CoreCraftProfessional Work43.166.1+23.0
Terminal-Bench 4.0Code44.463.6+19.2
Legal Research Bench ImmigrationLegal36.418.2−18.2
ALE-BenchCode26372132−17.9
τ³-Banking (Mercor run, bm25_grep retrieval)Agents43.059.8+16.8
Legal Research Bench Administrative/RegulatoryLegal39.456.1+16.7
τ³-Banking (Fireworks, banking_knowledge)Agents47.063.3+16.3
Harvey LAB held-out set (Fireworks run)Legal0.715.4+14.7
GDPval-AA v2.1Professional Work15141846+14.6
BrokenArXiv 08/2026 (Native Harnesses)Math78.664.3−14.3
Mystery Game PuzzlesGames56.042.0−14.0
Legal Research Bench HealthLegal60.073.3+13.3
Legal Research Bench Constitutional/Civil RightsLegal18.431.6+13.2
GBENCH One-shot Coding GScoreGames70.158.2−11.9
AutomationBench-AA 1.0.6Agents61.773.3+11.6
FrontierMath Tiers 1-3 (v2) - PrivateMath89.878.2−11.6
Legal Research Bench Civil LitigationLegal18.529.6+11.1
AA-Briefcase v1.1 EloAgents14761678+11.1
Chess Puzzles (task version 2.0)Games56.045.0−11.0
Tax Agent BenchFinance & Accounting53.063.9+10.8
Terminal-Bench-Science 0.1Science27.638.1+10.5
Design Arena Overall Code CategoriesCode12931353+10.0
DualEntry Accounting AI BenchmarkFinance & Accounting76.286.1+9.9
Harvey LAB (Mercor run, undisclosed ~213-task cohort)Legal4.113.8+9.7
WCB (SecureBio)Science60.551.5−9.0
DeepSWE v1.1Code66.675.5+8.9
ProofBench v1.1Math83.091.0+8.0
Legal Research Bench Weighted PassLegal79.271.4−7.7
APEX-Agents 1.1Professional Work68.775.9+7.2
LAB-Bench CloningScenarios (SecureBio)Science98.591.5−7.0
GPQA-Bio (SecureBio)Science88.982.2−6.7
NoCoT-Bench NCRI 15.2General QA115103−6.5
BuseyBench SVGMultimodal7.56.9−5.9
Humanity's Last Exam DiamondGeneral QA45.239.4−5.8
LiveBench 2.0Agents81.976.7−5.2
ACCoRDGeneral QA82.477.5−4.9
VCT (SecureBio)Science54.049.4−4.6
GameBench 2Games79.484.0+4.6
LMArena Text Arena EloChat & Writing14571483+4.6
Legal Research Bench FamilyLegal9.113.6+4.5
Conceptual Reasoning IndexIndexes71.067.8−4.4
HPCT (SecureBio)Science67.863.5−4.3
MysteryMechanismScience30.226.1−4.1
DTBenchGeneral QA97.393.3−4.0
LMArena Agent Arena Net Improvement (Steering Burden)Agents6.94.9−3.8
SRE Bench — Fully SolvedCode30.527.1−3.4
DeepsecBench — Revalidated V1 (233 issues)Code40.737.5−3.3
VCT v2 (SecureBio)Science49.346.2−3.1
LAB-Bench ProtocolQA (SecureBio)Science81.178.1−3.0
Vibe Code Bench v1.1Code87.884.9−2.9
Creative Writing v3 Rubric ScoreChat & Writing16.116.6+2.9
Longform Creative WritingChat & Writing82.879.9−2.9
APEX-Accounting 1.1Finance & Accounting59.762.4+2.7
DAYJOB: Finance (October 2026 rescoring)Finance & Accounting19.517.0−2.5
Furniture Assembly BenchmarkMultimodal58.355.8−2.5
Creative Writing v3 EloChat & Writing21262173+2.4
ZeroBench (Main questions, pass@5)Multimodal41.039.0−2.0
LMArena Agent Arena Bash ReliabilityAgents10.97.2−1.9
Legal Research Bench CriminalLegal26.328.1+1.8
LMArena Agent Arena Steering BurdenAgents10.16.7−1.7
Harvey LAB: In-House Contracting (held-out)Legal1.33.0+1.7
NoCoT-Bench NCKI (kspine v3)General QA119116−1.7
Epoch Capabilities Index (ECI)Indexes164162−1.6
MMMU ProMultimodal82.584.1+1.6
SimpleQA Verified (Epoch anti-abstention)General QA60.762.2+1.5
LMCAGeneral QA59.157.6−1.5
Terminal-Bench Science 0.1 (Vals)Science30.031.4+1.4
LAB-Bench LitQA2 (SecureBio)Science51.853.1+1.3
BigFinanceBench private holdout (final-answer accuracy)Finance & Accounting47.148.2+1.1
LMArena Agent Arena Praise vs ComplaintAgents10.08.0−1.0
ZeroBench Main Mean pass@1 (five samples)Multimodal28.829.8+1.0
LMArena Agent Arena Confirmed SuccessAgents4.52.7−0.9
BioMysteryBench (Vals Terminus 2)Science74.875.6+0.7
IOI (Vals v2)Code82.681.9−0.7
Vals Science IndexIndexes45.044.4−0.6
Epiq LAER CorporateBenchProfessional Work79.078.4−0.5
MBCT (SecureBio)Science61.160.7−0.4
MMLU-Bio (SecureBio)Science90.891.2+0.4
PubMedQA (SecureBio)Health & Medicine80.981.2+0.3
WMDP-Bio (SecureBio)Science90.089.7−0.3
Bio Capabilities Index (SecureBio)Indexes161161−0.1
LMArena Agent Arena Tool HallucinationAgents1.01.00.0
Legal Research Bench Business/CommercialLegal27.927.90.0
OTIS Mock AIME 2024-2025Math100.0100.00.0

GPT-6 SolvsKimi K3.1 Pronormalised score marginsData Nov 15, 2026Data: Vals AI, SecureBio, Epoch AI and 20 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…