Atlas

Compare Models

GPT-5.6 Sol vs Kimi K3.1 Pro

Kimi K3.1 Pro leads by 1.0 capability points (≈95% interval −1.3 to +3.2): statistically tied · P(higher) ≈ 80%

GPT-5.6 Sol wins 44 of 91 shared benchmarks (43 lost, 4 tied)

Capability vs. relative cost
GPT-5.6 SolKimi K3.1 ProClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

GPT-5.6 SolvsKimi K3.1 Procapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins44 – 43 · 91 shared, 4 tied

Comparing matching setups only (44 benchmarks excluded) · suites counted once
← GPT-5.6 SolKimi K3.1 Pro →

GPT-5.6 SolvsKimi K3.1 Prowins by categoryData Nov 15, 2026Data: Vals AI, SecureBio, Moonshot AI and 21 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Kimi K3.1 Pro minus GPT-5.6 Sol.

τ³-Banking (Fireworks, banking_knowledge)Agents35.763.3+27.7
Legal Research Bench ImmigrationLegal45.518.2−27.3
Terminal-Bench 4.0Code37.963.6+25.8
Legal Research Bench Business/CommercialLegal53.527.9−25.6
RiemannBench (November 2026 rescoring)Math83.260.8−22.4
EnterpriseBench: CoreCraftProfessional Work46.266.1+19.9
FrontierMath Tier 4 (v2) - PrivateMath82.965.9−17.1
Legal Research Bench Weighted PassLegal87.871.4−16.4
Mystery Game PuzzlesGames58.042.0−16.0
Legal Research Bench CriminalLegal43.928.1−15.8
Terminal-Bench-Science 0.1Science22.438.1+15.7
Harvey LAB held-out set (Fireworks run)Legal0.615.4+14.9
Chess Puzzles (task version 2.0)Games59.045.0−14.0
AutomationBench-AA 1.0.6Agents60.173.3+13.2
ExploitGym - release-reported percentage, cohort unspecifiedCybersecurity33.720.7−13.0
WCB (SecureBio)Science64.451.5−12.9
HPCT (SecureBio)Science76.363.5−12.8
Terminal-Bench Science 0.1 (Vals)Science20.031.4+11.4
Legal Research Bench Civil LitigationLegal40.729.6−11.1
Concept Synth INDUCTION Challenge64 Round 1General QA57.846.9−10.9
AA-Briefcase v1.1 EloAgents14791678+10.9
FrontierMath Tiers 1-3 (v2) - PrivateMath89.178.2−10.9
Chartography (with tools)Multimodal79.990.2+10.3
Creative Writing v3 EloChat & Writing19742173+10.1
GDPval-AA v2.1Professional Work16191846+10.0
ALE-BenchCode24102132−9.8
GBENCH One-shot Coding GScoreGames67.758.2−9.5
BenchCAD - geometric overlapMultimodal83.492.7+9.3
IOI (Vals v2)Code91.281.9−9.3
Concept Synth INDUCTION Challenge100 Round 1General QA43.034.0−9.0
Humanity's Last Exam DiamondGeneral QA30.439.4+9.0
PerceptionBenchMultimodal59.768.3+8.6
ProgramBench - almost solved rollout percentageCode23.031.5+8.5
BuseyBench SVGMultimodal6.16.9+8.3
ProofBench v1.1Math83.091.0+8.0
ZeroBench Main Mean pass@1 (five samples)Multimodal22.029.8+7.8
LAB-Bench CloningScenarios (SecureBio)Science98.891.5−7.3
MysteryMechanismScience33.326.1−7.2
Legal Research Bench HealthLegal80.073.3−6.7
BabyVision with PythonMultimodal88.995.4+6.5
Humanity's Last ExamGeneral QA45.551.1+5.6
Bio Capabilities Index (SecureBio)Indexes169161−5.5
DAYJOB: Finance (October 2026 rescoring)Finance & Accounting11.817.0+5.2
VCT (SecureBio)Science54.349.4−4.9
Legal Research Bench Administrative/RegulatoryLegal60.656.1−4.5
BioMysteryBench (Vals Terminus 2)Science71.175.6+4.4
CharXiv ReasoningMultimodal84.689.0+4.4
Vibe Code Bench v1.1Code80.584.9+4.4
Tax Agent BenchFinance & Accounting68.063.9−4.1
BigFinanceBench private holdout (final-answer accuracy)Finance & Accounting44.248.2+4.0
GameBench 2Games80.184.0+4.0
LAB-Bench ProtocolQA (SecureBio)Science81.978.1−3.8
NoCoT-Bench NCRI 15.2General QA109103−3.6
SRE Bench — Fully SolvedCode30.527.1−3.4
ORCA-bench Hard RCA (Fireworks 324-task subset)Agents29.532.9+3.4
VCT v2 (SecureBio)Science49.446.2−3.2
GPQA-Bio (SecureBio)Science85.382.2−3.1
NoCoT-Bench NCKI (kspine v3)General QA121116−3.0
Vals Science IndexIndexes41.544.4+2.9
DeepSWE v1.1Code72.775.5+2.8
Legal Research Bench Constitutional/Civil RightsLegal34.231.6−2.6
DTBenchGeneral QA95.793.3−2.4
Creative Writing v3 Rubric ScoreChat & Writing16.216.6+2.3
DeepsecBench — Revalidated V1 (233 issues)Code35.337.5+2.2
LMArena Agent Arena Bash ReliabilityAgents11.47.2−2.1
Longform Creative WritingChat & Writing81.779.9−1.8
ACCoRDGeneral QA75.977.5+1.6
Conceptual Reasoning IndexIndexes68.767.8−1.2
Harvey LAB: In-House Contracting (held-out)Legal2.03.0+1.0
LMCAGeneral QA58.557.6−0.9
MMMU ProMultimodal83.083.9+0.9
Furniture Assembly BenchmarkMultimodal56.755.8−0.8
LMArena Agent Arena Praise vs ComplaintAgents6.38.0+0.8
Epoch Capabilities Index (ECI)Indexes163162−0.8
Political Science LLM BenchmarkProfessional Work69.268.6−0.7
LMArena Agent Arena Steering BurdenAgents5.56.7+0.6
Prophet Arena Arena Score (Fixed Context)Forecasting84.885.4+0.6
LMArena Agent Arena Net Improvement (Steering Burden)Agents4.64.9+0.5
LiveBench 2.0Agents76.376.7+0.5
ABC-Bench liquid handling (SecureBio)Science98.798.3−0.4
MBCT (SecureBio)Science61.160.7−0.4
LMArena Text Arena EloChat & Writing14851483−0.3
LAB-Bench LitQA2 (SecureBio)Science52.853.1+0.3
LMArena Agent Arena Confirmed SuccessAgents2.32.7+0.2
BrowseCompSearch92.292.1−0.1
MMLU-Bio (SecureBio)Science91.191.2+0.1
PubMedQA (SecureBio)Health & Medicine81.181.2+0.1
LMArena Agent Arena Tool HallucinationAgents1.01.00.0
Legal Research Bench FamilyLegal13.613.60.0
OTIS Mock AIME 2024-2025Math100.0100.00.0
WMDP-Bio (SecureBio)Science89.789.70.0

GPT-5.6 SolvsKimi K3.1 Pronormalised score marginsData Nov 15, 2026Data: Vals AI, SecureBio, Moonshot AI and 21 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…