Atlas

Compare Models

GPT-6.1 Sol vs GPT-6 Sol

GPT-6.1 Sol leads by 2.5 capability points (≈95% interval +0.1 to +5.0): a clear lead over GPT-6 Sol · P(higher) ≈ 98%

Wins 183 of 216 shared benchmarks (28 lost, 5 tied)

Capability vs. relative cost
GPT-6.1 SolGPT-6 SolClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

GPT-6.1 SolvsGPT-6 Solcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins183 – 28 · 216 shared, 5 tied

Comparing matching setups only (15 benchmarks excluded) · suites counted once
← GPT-6.1 SolGPT-6 Sol →

GPT-6.1 SolvsGPT-6 Solwins by categoryData Nov 15, 2026Data: Artificial Analysis, Mercor, Vals AI and 60 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6 Sol minus GPT-6.1 Sol.

ARC-AGI-3 (Semi-Private)General QA96.223.0−73.1
AA-PersonalityBench Organization indexChat & Writing81.012.4−69.2
Short Form Bench v1.0Games29492394−55.0
AA-PersonalityBench Transparency indexChat & Writing92.439.1−53.8
Clipping Bench v1.0Games57071240−48.1
Search ArenaSearch12301174−44.8
CyberBench v1.1 OverallCybersecurity39.378.0+38.7
AA-PersonalityBench Imagination indexChat & Writing61.929.6−32.7
Terminal-Bench-Science 0.1Science58.130.0−28.1
FrontierCUA pass@1Agents46.720.0−26.7
Mystery Game PuzzlesGames80.056.0−24.0
Terminal-Bench Science 0.1 (Vals)Science52.930.0−22.9
Furniture Assembly BenchmarkMultimodal80.058.3−21.7
SRE Bench — Fully SolvedCode50.830.5−20.2
NoCoT-Bench NCRI 15.2General QA149115−18.9
WeirdML v3Science38.319.7−18.5
Medical Long Context Reasoning (MLCR-AA)Health & Medicine33.916.1−17.8
YC-BenchGames9362671441793+17.6
AA-PersonalityBench Expressiveness indexChat & Writing50.533.4−17.3
AA-PersonalityBench Informativeness indexChat & Writing27.710.5−17.3
MysteryMechanismScience46.430.2−16.2
ExploitBench - Internal Port (June-August 2026)Cybersecurity21.55.5−16.0
ProofBench v1.1Math99.083.0−16.0
ChessBench EloGames15621303−15.4
Humanity's Sixth SenseMultimodal46.631.2−15.4
SpireClimbGames23.89.2−14.7
Vals Science IndexIndexes59.645.0−14.6
Bug Hunt BenchCode42.227.9−14.3
IOI (Vals v2)Code96.982.6−14.3
Humanity's Last Exam DiamondGeneral QA58.845.2−13.6
AA-PersonalityBench Slop indexChat & Writing14.31.0−13.5
SimpleQA Verified (Epoch anti-abstention)General QA73.960.7−13.2
OpenAI internal computer-use safety - adverse outcome rateSafety4.317.4−13.1
ExploitGym - release-reported percentage, cohort unspecifiedCybersecurity35.122.1−13.0
Hermes IndexIndexes57.144.1−13.0
Decision-Bench v1.1Professional Work49.136.5−12.6
GameBench 2Games92.079.4−12.5
ALE-BenchCode29902637−12.5
Terminal-Bench 4.0 (Artificial Analysis)Code56.143.9−12.1
Chess Puzzles (task version 2.0)Games68.056.0−12.0
AA-PersonalityBench Warmth indexChat & Writing65.754.3−11.5
KernelBench Mega Normalized (RTX PRO 6000)Code14.12.7−11.4
HealthBench Hard (updated implementation)Health & Medicine33.422.1−11.3
VoxelBench Text RatingGames25502394−11.1
BullshitBench v2 LeaderboardSafety65.054.0−11.0
LMArena Agent Arena Net Improvement (Steering Burden)Agents12.56.9−10.5
Artificial Analysis Intelligence Index v5.0.1Indexes54.644.2−10.5
Design Arena Overall Code CategoriesCode13561293−10.4
FrontierMath Tier 4 (v2) - PrivateMath100.090.0−10.0
TaxCalcBench TY25Finance & Accounting38.028.0−10.0
WellnessBenchSafety66.056.0−10.0
SimpleBenchGeneral QA82.973.1−9.8
Legal Research BenchLegal38.528.8−9.6
LisanBench Difficulty-WeightedGames40043118−9.5
TxBench Oligo Discovery V1 (113 evaluations)Science52.543.1−9.4
MMMU-Pro Extended (Mercor held-out)Multimodal61.351.9−9.4
Professional Reasoning Benchmark - LegalLegal47.938.6−9.3
Tax Agent BenchFinance & Accounting62.353.0−9.3
LMArena Code Arena WebDevCode17431673−9.2
GBENCH One-shot Coding GScoreGames79.070.1−8.9
Arena Image-to-WebDevCode16961655−8.8
Terminal-Bench 4.0Code58.249.4−8.8
LMArena Agent Arena Praise vs ComplaintAgents27.510.0−8.8
HardGeoBench <=10miMultimodal35.026.3−8.7
DAYJOB: FinanceFinance & Accounting18.09.3−8.7
HealthBenchHealth & Medicine56.948.2−8.7
DeepSWE v1.1Code75.266.6−8.6
SpatialBench-Long V1 Clean RerunsScience39.931.6−8.3
Toloka Arena (v3)Agents66.058.0−8.0
MazeBench ASCIIGames9.01.0−8.0
HardGeoBench Avg DistanceMultimodal12391497−8.0
Code MigrationCode65.157.2−7.9
NoCoT-Bench NCKI (kspine v3)General QA130119−7.5
BioSecBench-Structure V1 (58 evaluations)Science21.313.8−7.5
VariantBench V1 Clean RerunsScience46.038.7−7.3
AA-Omniscience IndexIndexes41.527.1−7.2
EnterpriseBench: CoreCraftProfessional Work50.343.1−7.2
HealthBench ProfessionalHealth & Medicine68.361.2−7.1
GeneBench-Pro Public Release SubsetScience33.026.0−7.0
SWE-bench VerifiedCode87.380.3−7.0
OSWorld 2 (Aug 8) offline - reported partial reward, step limit unspecifiedAgents71.464.4−7.0
GDP.pdf (Artificial Analysis, All-pass)Professional Work32.025.2−6.8
BioSecBench-Function V1 Clean RerunsScience46.039.3−6.8
ChessBenchGames68.762.0−6.7
RuneBenchGames6.96.4−6.6
Humanity's Last Exam (Text-Only)General QA52.646.3−6.3
TxBench Antibody Discovery V1 (100 evaluations)Science49.743.3−6.3
EpiBench V1 Clean RerunsScience28.322.0−6.3
HardGeoBench <=100miMultimodal43.837.5−6.2
HardGeoBench Country AccuracyMultimodal52.546.3−6.2
DistressBenchSafety74.268.0−6.2
HealthBench Hard (length-adjusted)Health & Medicine36.230.1−6.1
BrowseComp (Mercor 130-question subset)Search95.489.4−6.0
SWE-bench Verified Extended (Mercor held-out)Code45.339.3−6.0
Financial Coaching BenchFinance & Accounting68.562.6−5.9
RCM-Bench-1Health & Medicine87.481.5−5.8
Bedside BenchHealth & Medicine93.988.1−5.8
Extended NYT Connections 940 (Quadratic v1)Games95.590.1−5.4
HieroglyphBenchMultimodal31.426.0−5.4
FrontierPhysics v0.1Science19.614.3−5.3
Artificial Analysis Progress Index v1Indexes348334−5.1
ArXivMath 05/2026Math95.090.0−5.0
DualEntry Accounting AI BenchmarkFinance & Accounting81.276.2−5.0
SUPERChemScience77.872.8−5.0
BigFinanceBench private holdout (final-answer accuracy)Finance & Accounting52.047.1−4.9
GDP.xlsxProfessional Work22.017.1−4.9
AutomationBench-AA 1.0.6Agents66.661.7−4.9
BioMysteryBench (Vals Terminus 2)Science79.674.8−4.8
Roboflow Vision Evals (October 2026, five tasks)Multimodal81.776.9−4.8
FrontierCode 2.0 - ScoreCode39.634.9−4.7
FrontierCode 2.0 - Pass RateCode43.038.3−4.7
Harvey LAB (Mercor run, undisclosed ~213-task cohort)Legal8.84.1−4.7
BrokenArXiv 06/2026Math100.095.4−4.6
APEX-Accounting 1.1Finance & Accounting64.359.7−4.6
ARC-AGI-2General QA94.289.6−4.6
APEX-Agents 1.1Professional Work73.268.7−4.5
BrowseComp Extended (Mercor held-out)Search67.763.2−4.5
MedScribeHealth & Medicine86.582.0−4.4
LMArena Text Arena EloChat & Writing14821457−4.4
MedXpertQA MM (200-question set)Health & Medicine85.781.3−4.4
AA-Briefcase v1.1 EloAgents15551476−4.3
SWE Atlas - Codebase QnACode48.444.1−4.3
Artificial Analysis Intelligence Index v4.3.2Indexes51.847.6−4.2
BuseyBench SVGMultimodal8.07.5−4.2
GDPval-AA v2.1Professional Work16071514−4.1
Ruby LLM Benchmarks — Program FixerCode75.079.0+4.0
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine24.020.0−4.0
DAYJOB: HealthcareHealth & Medicine7.63.6−4.0
Epoch Capabilities Index (ECI)Indexes169164−3.9
FrontierMath Tiers 1-3 (v2) - PrivateMath93.789.8−3.9
AA-PersonalityBench Conviction indexChat & Writing54.350.5−3.8
Harvey's Legal Agent BenchmarkLegal5.41.7−3.7
LMCAGeneral QA62.859.1−3.7
MentalHealthBenchSafety57.954.2−3.7
MetagenomicsBench V1Science45.742.0−3.7
Vals Index v2.1Indexes61.257.5−3.6
LMArena Agent Arena Confirmed SuccessAgents11.74.5−3.6
ChessBench AccuracyGames80.777.4−3.5
LiveBench 2.0Agents85.481.9−3.5
Artificial Analysis Coding Agent Index v1.5Indexes60.156.7−3.5
ArXivMath 06/2026Math94.491.0−3.5
Terminal-Bench 2.1 Extended (Mercor held-out)Code45.842.4−3.4
Harvey LAB: In-House Contracting (held-out)Legal4.71.3−3.3
Harvey LAB-AA v1.1 (hallucination-gated)Legal6.93.6−3.3
TxBench Preclinical Pharmacology V1 Clean RerunsScience59.763.0+3.3
MedXpertQA MM Extended (Mercor held-out)Health & Medicine64.768.0+3.3
Epiq LAER CorporateBenchProfessional Work82.279.0−3.2
Roboflow Vision Evals (September 2026 protocol)Multimodal85.582.3−3.2
SWE-bench MultilingualCode84.981.7−3.2
BioSecBench-Surveillance V1 Clean RerunsScience50.847.7−3.0
ARC-AGI-1 Semi-PrivateGeneral QA98.595.5−3.0
BrokenArXiv 05/2026Math90.087.0−3.0
LMArena Agent Arena Bash ReliabilityAgents5.010.9+3.0
Finance Agent v2Finance & Accounting52.049.0−3.0
SM BenchSafety69.066.0−3.0
FrontierMath ErdősMath4.41.5−2.9
AutomationBench 1.0.6 (provider-reported)Agents36.133.2−2.9
FACTS Benchmark SuiteGeneral QA75.372.5−2.8
LLM Chess EloGames16121545−2.7
Public Benefits Bench v1.1Professional Work59.356.6−2.7
MMMU ProMultimodal85.182.5−2.5
CVDP (517-problem set)Code55.152.7−2.4
τ³-Banking (Fireworks, banking_knowledge)Agents44.747.0+2.3
SWE-TogetherCode58.360.6+2.3
Conceptual Reasoning IndexIndexes72.771.0−2.3
MMLU ProGeneral QA91.088.9−2.1
ScBench V1 Clean RerunsScience63.661.5−2.1
SciCode (main problems)Science14.612.7−1.9
APEX-SWECode46.945.0−1.9
SciCode 1.0.1 (Artificial Analysis)Code55.857.6+1.9
MacroscopeBenchCode78.580.3+1.8
MedCodeHealth & Medicine48.847.1−1.8
SAGEMath46.544.8−1.7
CritPt v2 (agentic, avg@5)Science82.680.9−1.7
ScBench-Long V1 Clean RerunsScience26.728.4+1.7
BioSimBench 1.0Science15.313.6−1.7
LMArena Agent Arena Steering BurdenAgents13.410.1−1.6
LisanBench Path LengthGames87368222−1.6
ACCoRDGeneral QA80.982.4+1.5
VulcanBench Frontier v4Code88.386.8−1.5
AdvancedIFChat & Writing82.480.9−1.5
Gray Swan IPI Attack Success (k=15)Safety8.610.1−1.5
Vals RSI Index v1.1 (four tasks)Agents26.628.1+1.5
AA-3D v1.0 EloCode11211131+1.5
CharXiv Extended (Mercor held-out)Multimodal85.183.8−1.3
DeepsecBench — Revalidated V1 (233 issues)Code39.540.7+1.3
LMArena Agent Arena Bash RecoveryAgents4.57.0+1.3
KernelBench CUDA Normalized (RTX PRO 6000)Code43.242.0−1.2
SpeciEval (Pooled English Runs)Safety97.498.5+1.1
SpeciEval (Refusal-Aware Retries, Historical)Safety97.498.5+1.1
Vibe Code Bench v1.1Code88.987.8−1.1
Olam Social Poker EloGames15431542−1.1
CharXiv (descriptive and reasoning combined)Multimodal96.095.0−1.0
ProgramBenchCode3.02.0−1.0
EBR-bench v4 (card ban, single agent)Games54.353.3−1.0
ForecastBench Preliminary Dataset Brier IndexForecasting61.762.6+0.9
FrontierCode 1.1 Extended - ScoreCode59.860.7+0.9
CritPt (avg@5)Science31.730.9−0.9
DTBenchGeneral QA96.597.3+0.8
Excel Modeling BenchmarkFinance & Accounting70.871.5+0.8
ChessBench CoherenceGames87.588.2+0.7
ObviousBenchGeneral QA100.099.3−0.7
SWE-Marathon v1.1Code42.543.1+0.6
Wiz Cyber Model Arena v3.9Cybersecurity70.671.2+0.6
FrontierCode 1.1 Extended - Pass RateCode65.966.3+0.5
Hermes BenchAgents73.973.5−0.4
Arena Alignment IndexSafety87.987.6−0.3
AA-LCR v1.1Search84.083.7−0.3
BioSecBench-Refusal V1 Clean RerunsSafety37.637.9+0.3
SpatialBench V1 Clean RerunsScience67.367.5+0.3
LMArena Agent Arena Tool HallucinationAgents1.01.00.0
AIME 2025Math100.0100.00.0
OTIS Mock AIME 2024-2025Math100.0100.00.0
SWE Atlas Codebase QnA Extended (Mercor held-out)Code31.331.30.0
Terminal-Bench 2.1Code85.485.40.0
τ³-Banking (Mercor run, bm25_grep retrieval)Agents43.043.00.0

GPT-6.1 SolvsGPT-6 Solnormalised score marginsData Nov 15, 2026Data: Artificial Analysis, Mercor, Vals AI and 60 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…