Atlas

Compare Models

GPT-6.1 Sol vs Kimi K3.1 Pro

GPT-6.1 Sol leads by 2.4 capability points (≈95% interval −0.0 to +4.9): statistically tied · P(higher) ≈ 97%

Wins 59 of 87 shared benchmarks (22 lost, 6 tied)

Capability vs. relative cost
GPT-6.1 SolKimi K3.1 ProClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

GPT-6.1 SolvsKimi K3.1 Procapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins59 – 22 · 87 shared, 6 tied

Comparing matching setups only (112 benchmarks excluded) · suites counted once
← GPT-6.1 SolKimi K3.1 Pro →

GPT-6.1 SolvsKimi K3.1 Prowins by categoryData Nov 15, 2026Data: Vals AI, Epoch AI, Arena and 23 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Kimi K3.1 Pro minus GPT-6.1 Sol.

Concept Synth INDUCTION Challenge100 Round 1General QA90.034.0−56.0
Concept Synth INDUCTION Challenge64 Round 1General QA95.346.9−48.4
Mystery Game PuzzlesGames80.042.0−38.0
FrontierMath Tier 4 (v2) - PrivateMath100.065.9−34.1
ALE-BenchCode29902132−30.4
OpenWorldEngineer (Mean@8)Code41.213.6−27.6
NoCoT-Bench NCRI 15.2General QA149103−25.4
Furniture Assembly BenchmarkMultimodal80.055.8−24.2
SRE Bench — Fully SolvedCode50.827.1−23.7
Chess Puzzles (task version 2.0)Games68.045.0−23.0
Terminal-Bench Science 0.1 (Vals)Science52.931.4−21.4
GBENCH One-shot Coding GScoreGames79.058.2−20.8
MysteryMechanismScience46.426.1−20.3
Humanity's Last Exam DiamondGeneral QA58.839.4−19.4
τ³-Banking (Fireworks, banking_knowledge)Agents44.763.3+18.6
RiemannBench (November 2026 rescoring)Math79.260.8−18.4
sudo L7 (October 2026 rescoring)Code33.916.7−17.2
τ³-Banking (Mercor run, bm25_grep retrieval)Agents43.059.8+16.8
DAYJOB: Finance (October 2026 rescoring)Finance & Accounting33.517.0−16.5
EnterpriseBench: CoreCraftProfessional Work50.366.1+15.8
Code Migration v1.1 (Vals Index v2.2 subset)Code75.159.7−15.5
FrontierMath Tiers 1-3 (v2) - PrivateMath93.778.2−15.4
Vals Science IndexIndexes59.644.4−15.3
IOI (Vals v2)Code96.981.9−15.0
LMArena Agent Arena Net Improvement (Steering Burden)Agents12.54.9−14.3
DAYJOB: Data ScienceProfessional Work18.74.7−14.0
Legal Research Bench Weighted PassLegal84.371.4−12.9
HoneyBench v0.2 (hack rate)Safety59.571.8−12.3
SimpleQA Verified (Epoch anti-abstention)General QA73.962.2−11.7
Legal Research Bench Business/CommercialLegal39.527.9−11.6
DAYJOB: AccountingFinance & Accounting25.213.6−11.6
GDPval-AA v2.1Professional Work16071846+10.5
BuseyBench SVGMultimodal8.06.9−10.1
LMArena Agent Arena Praise vs ComplaintAgents27.58.0−9.8
NoCoT-Bench NCKI (kspine v3)General QA130116−9.2
Visa Agentic Commerce BenchmarkAgents62.253.3−8.9
LiveBench 2.0Agents85.476.7−8.7
Terminal-Bench 4.0Code55.163.6+8.6
ProofBench v1.1Math99.091.0−8.0
Legacy-Bench (Fireworks, avg@3)Code55.063.0+8.0
GameBench 2Games92.084.0−7.9
DAYJOB: LawLegal31.823.9−7.9
Legal Research Bench CriminalLegal35.128.1−7.0
AA-Briefcase v1.1 EloAgents15551678+6.8
AutomationBench-AA 1.0.6Agents66.673.3+6.7
Conceptual Reasoning IndexIndexes72.767.8−6.7
DAXBench v2Code92.686.2−6.4
Epoch Capabilities Index (ECI)Indexes169162−5.5
HLAB-Vals-1.1Legal6.311.7+5.4
LMCAGeneral QA62.857.6−5.2
Harvey LAB (Mercor run, undisclosed ~213-task cohort)Legal8.813.8+5.0
DualEntry Accounting AI BenchmarkFinance & Accounting81.286.1+4.9
Legal Research Bench FamilyLegal18.213.6−4.5
LMArena Agent Arena Confirmed SuccessAgents11.72.7−4.5
FrontierMemory v1.1Agents51.155.6+4.4
BioMysteryBench (Vals Terminus 2)Science79.675.6−4.1
Vibe Code Bench v1.1Code88.984.9−4.0
BigFinanceBench private holdout (final-answer accuracy)Finance & Accounting52.048.2−3.8
Epiq LAER CorporateBenchProfessional Work82.278.4−3.8
Access to Justice BenchmarkLegal65.869.5+3.6
ACCoRDGeneral QA80.977.5−3.4
Legal Research Bench HealthLegal76.773.3−3.3
LMArena Agent Arena Steering BurdenAgents13.46.7−3.3
DTBenchGeneral QA96.593.3−3.2
Legal Research Bench Administrative/RegulatoryLegal53.056.1+3.0
APEX-Agents 1.1Professional Work73.275.9+2.7
DeepsecBench — Revalidated V1 (233 issues)Code39.537.5−2.0
APEX-Accounting 1.1Finance & Accounting64.362.4−1.9
Harvey LAB: In-House Contracting (held-out)Legal4.73.0−1.7
Tax Agent BenchFinance & Accounting62.363.9+1.6
LMArena Agent Arena Bash ReliabilityAgents5.07.2+1.1
Excel Modeling Benchmark v1.1Finance & Accounting71.170.0−1.1
MMMU ProMultimodal85.184.1−1.0
Prophet Arena Arena Score (Fixed Context)Forecasting84.885.4+0.6
Design Arena Overall Code CategoriesCode13561353−0.5
Finance Agent v2.1Finance & Accounting61.861.5−0.3
DeepSWE v1.1Code75.275.5+0.3
Vals Finance IndexIndexes55.856.0+0.2
Humanity's Last ExamGeneral QA51.351.1−0.2
LMArena Text Arena EloChat & Writing14821483+0.2
Prophet Arena Arena Score (Agentic, OpenCode)Forecasting86.086.1+0.1
LMArena Agent Arena Tool HallucinationAgents1.01.00.0
APEX-1 (Mercor, mean score)Professional Work69.369.30.0
Legal Research Bench Civil LitigationLegal29.629.60.0
Legal Research Bench Constitutional/Civil RightsLegal31.631.60.0
Legal Research Bench ImmigrationLegal18.218.20.0
OTIS Mock AIME 2024-2025Math100.0100.00.0

GPT-6.1 SolvsKimi K3.1 Pronormalised score marginsData Nov 15, 2026Data: Vals AI, Epoch AI, Arena and 23 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…