Atlas

Compare Models

GPT-6 Astra vs Kimi K3.1 Pro

GPT-6 Astra leads by 3.4 capability points (≈95% interval +0.9 to +5.9): a clear lead over Kimi K3.1 Pro · P(higher) ≈ 100%

Wins 80 of 107 shared benchmarks (24 lost, 3 tied)

Capability vs. relative cost
GPT-6 AstraKimi K3.1 ProClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

GPT-6 AstravsKimi K3.1 Procapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins80 – 24 · 107 shared, 3 tied

Comparing matching setups only (142 benchmarks excluded) · suites counted once
← GPT-6 AstraKimi K3.1 Pro →

GPT-6 AstravsKimi K3.1 Prowins by categoryData Nov 15, 2026Data: Vals AI, SecureBio, Epoch AI and 27 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Kimi K3.1 Pro minus GPT-6 Astra.

Concept Synth INDUCTION Challenge100 Round 1General QA94.034.0−60.0
Concept Synth INDUCTION Challenge64 Round 1General QA98.446.9−51.6
Mystery Game PuzzlesGames84.042.0−42.0
ALE-BenchCode31642098−37.8
FrontierMath Tier 4 (v2) - PrivateMath97.665.9−31.7
Terminal-Bench Science 0.1 (Vals)Science62.931.4−31.4
NoCoT-Bench NCRI 15.2General QA159103−30.9
OpenWorldEngineer (Mean@8)Code44.213.6−30.6
SRE Bench — Fully SolvedCode56.927.1−29.8
Chess Puzzles (task version 2.0)Games74.045.0−29.0
MysteryMechanismScience53.226.1−27.0
Terminal-Bench-Science 0.1Science64.638.1−26.5
Humanity's Last Exam DiamondGeneral QA63.839.4−24.4
Furniture Assembly BenchmarkMultimodal80.055.8−24.2
CloudOpsBench v1.0Agents42.819.1−23.7
EnterpriseBench: CoreCraftProfessional Work42.666.1+23.5
ABC-Bench liquid handling (SecureBio)Science76.798.3+21.6
GBENCH One-shot Coding GScoreGames79.258.2−21.0
HoneyBench v0.2 (hack rate)Safety50.871.8−21.0
RiemannBench (November 2026 rescoring)Math81.660.8−20.8
Vals Science IndexIndexes65.144.4−20.7
DAYJOB: Finance (October 2026 rescoring)Finance & Accounting37.317.0−20.3
τ³-Banking (Fireworks, banking_knowledge)Agents44.863.3+18.5
Code Migration v1.1 (Vals Index v2.2 subset)Code78.059.7−18.3
LMArena Agent Arena Net Improvement (Steering Burden)Agents14.54.9−18.2
Legal Research Bench ImmigrationLegal36.418.2−18.2
IOI (Vals v2)Code100.081.9−18.1
ORCA-bench Hard RCA (Fireworks 324-task subset)Agents50.232.9−17.3
DAYJOB: Data ScienceProfessional Work21.34.7−16.6
DAYJOB: AccountingFinance & Accounting30.013.6−16.4
FrontierMath Tiers 1-3 (v2) - PrivateMath93.778.2−15.4
sudo L7 (October 2026 rescoring)Code31.116.7−14.4
ZeroBench Main Mean pass@1 (five samples)Multimodal43.229.8−13.4
SimpleQA Verified (Epoch anti-abstention)General QA75.662.2−13.4
Legal Research Bench Weighted PassLegal84.871.4−13.3
τ³-Banking (Mercor run, bm25_grep retrieval)Agents46.759.8+13.1
DAYJOB: LawLegal36.923.9−13.0
ZeroBench (Main questions, pass@5)Multimodal52.039.0−13.0
LMArena Agent Arena Praise vs ComplaintAgents31.78.0−11.9
VCT v2 (SecureBio)Science58.046.2−11.8
GDPval-AA v2.1Professional Work15871846+11.4
micro1 PharmaBenchHealth & Medicine16.127.5+11.4
LAB-Bench ProtocolQA (SecureBio)Science89.378.1−11.2
Visa Agentic Commerce BenchmarkAgents64.453.3−11.1
Legal Research Bench Civil LitigationLegal40.729.6−11.1
WCB (SecureBio)Science62.651.5−11.1
DAXBench v2Code96.886.2−10.6
GameBench 2Games94.384.0−10.3
NoCoT-Bench NCKI (kspine v3)General QA131116−10.2
LiveBench 2.0Agents86.176.7−9.4
VCT (SecureBio)Science58.549.4−9.1
LAB-Bench CloningScenarios (SecureBio)Science100.091.5−8.5
Conceptual Reasoning IndexIndexes73.967.8−8.3
ProofBench v1.1Math99.091.0−8.0
Legacy-Bench (Fireworks, avg@3)Code55.763.0+7.3
LMCAGeneral QA64.457.6−6.8
BigFinanceBench private holdout (final-answer accuracy)Finance & Accounting54.648.2−6.4
AA-Briefcase v1.1 EloAgents15691678+6.0
Epoch Capabilities Index (ECI)Indexes169162−5.8
LMArena Agent Arena Confirmed SuccessAgents14.02.7−5.6
HPCT (SecureBio)Science69.063.5−5.5
Legal Research Bench Constitutional/Civil RightsLegal26.331.6+5.3
GPQA-Bio (SecureBio)Science87.482.2−5.2
DualEntry Accounting AI BenchmarkFinance & Accounting81.286.1+4.9
AutomationBench-AA 1.0.6Agents68.573.3+4.8
Legal Research Bench Business/CommercialLegal32.627.9−4.7
Harvey LAB (Mercor run, undisclosed ~213-task cohort)Legal9.213.8+4.6
HLAB-Vals-1.1Legal7.111.7+4.6
Legal Research Bench FamilyLegal18.213.6−4.5
ACCoRDGeneral QA81.777.5−4.2
Terminal-Bench 4.0Code59.663.6+4.0
LMArena Agent Arena Steering BurdenAgents14.86.7−4.0
DTBenchGeneral QA97.393.3−4.0
BuseyBench SVGMultimodal7.36.9−3.8
BioMysteryBench (Vals Terminus 2)Science79.375.6−3.7
Epiq LAER CorporateBenchProfessional Work82.078.4−3.5
Creative Writing v3 Rubric ScoreChat & Writing15.916.6+3.5
Legal Research Bench HealthLegal76.773.3−3.3
BenchCAD - geometric overlapMultimodal95.992.7−3.2
MBCT (SecureBio)Science63.960.7−3.2
Legal Research Bench Administrative/RegulatoryLegal53.056.1+3.0
Longform Creative WritingChat & Writing82.879.9−2.9
Excel Modeling Benchmark v1.1Finance & Accounting72.770.0−2.7
Humanity's Last ExamGeneral QA53.651.1−2.5
Harvey LAB: In-House Contracting (held-out)Legal5.33.0−2.3
MMMU ProMultimodal86.484.1−2.3
APEX-Accounting 1.1Finance & Accounting64.762.4−2.3
Bio Capabilities Index (SecureBio)Indexes164161−2.2
Legal Research Bench CriminalLegal29.828.1−1.8
LAB-Bench LitQA2 (SecureBio)Science54.853.1−1.7
LMArena Text Arena EloChat & Writing14741483+1.6
DeepSWE v1.1Code74.175.5+1.4
WMDP-Bio (SecureBio)Science90.889.7−1.1
Finance Agent v2.1Finance & Accounting60.461.5+1.1
VulnPR-100Code46.047.0+1.0
APEX-Agents 1.1Professional Work75.175.9+0.8
Prophet Arena Arena Score (Fixed Context)Forecasting84.785.4+0.7
Tax Agent BenchFinance & Accounting63.363.9+0.5
Vals Finance IndexIndexes55.656.0+0.4
LMArena Agent Arena Bash ReliabilityAgents7.87.2−0.3
Creative Writing v3 EloChat & Writing21772173−0.2
DeepsecBench — Revalidated V1 (233 issues)Code37.637.5−0.2
MMLU-Bio (SecureBio)Science91.391.2−0.1
Prophet Arena Arena Score (Agentic, OpenCode)Forecasting86.286.1−0.1
LMArena Agent Arena Tool HallucinationAgents1.01.00.0
OTIS Mock AIME 2024-2025Math100.0100.00.0
PubMedQA (SecureBio)Health & Medicine81.281.20.0

GPT-6 AstravsKimi K3.1 Pronormalised score marginsData Nov 15, 2026Data: Vals AI, SecureBio, Epoch AI and 27 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…