Atlas

Compare Models

GPT-5.6 Sol vs GPT-6.1 Sol

GPT-6.1 Sol leads by 3.4 capability points (≈95% interval +1.0 to +5.8): a clear lead over GPT-5.6 Sol · P(higher) ≈ 100%

Wins 173 of 216 shared benchmarks (39 lost, 4 tied)

Capability vs. relative cost
GPT-5.6 SolGPT-6.1 SolClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

GPT-5.6 SolvsGPT-6.1 Solcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins39 – 173 · 216 shared, 4 tied

Comparing matching setups only (17 benchmarks excluded) · suites counted once
← GPT-5.6 SolGPT-6.1 Sol →

GPT-5.6 SolvsGPT-6.1 Solwins by categoryData Nov 15, 2026Data: Vals AI, benchmarks.bio, Artificial Analysis and 65 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6.1 Sol minus GPT-5.6 Sol.

Short Form Bench v1.0Games24102949+53.4
ActiveVisionMultimodal23.571.8+48.3
Concept Synth INDUCTION Challenge100 Round 1General QA43.090.0+47.0
ARC-AGI-3 (Semi-Private)General QA7.852.7+44.9
Design Creativity BenchCode51.363.7+39.4
Concept Synth INDUCTION Challenge64 Round 1General QA57.895.3+37.5
CyberBench v1.1 OverallCybersecurity76.339.3−37.0
Terminal-Bench-Science 0.1Science22.458.1+35.7
BrowseComp Extended (Mercor held-out)Search32.367.7+35.4
Terminal-Bench Science 0.1 (Vals)Science20.052.9+32.9
BrokenArXiv 06/2026Math67.3100.0+32.7
CheatBenchSafety74.942.8+32.1
FrontierCUA pass@1Agents16.446.7+30.3
HiL-Bench (Human-in-Loop Benchmark)Agents32.362.3+30.0
Humanity's Last Exam DiamondGeneral QA30.458.8+28.4
LisanBench Difficulty-WeightedGames14034004+27.9
Clocktower BenchGames15831764+26.0
Furniture Assembly BenchmarkMultimodal56.780.0+23.3
WeirdML v3Science15.038.3+23.3
VIALSMultimodal26.549.1+22.6
Search ArenaSearch12581230−22.0
Mystery Game PuzzlesGames58.080.0+22.0
NoCoT-Bench NCRI 15.2General QA109149+21.7
Agent Red Teaming k=100 (CAIS)Safety40.318.8+21.6
CrossSearch (pass^3)Search38.059.0+21.0
ALE-BenchCode24102990+20.5
ChessBench EloGames12181562+20.5
TissueMicroscopyBench V1 (49 evaluations)Science2.022.4+20.4
SRE Bench — Fully SolvedCode30.550.8+20.2
Arena Image-to-WebDevCode16061696+19.5
TextQuests Game ProgressGames52.171.4+19.4
Gray Swan IPI Attack Success (k=15)Safety27.08.6+18.4
BuseyBench SVGMultimodal6.18.0+18.4
Vals Science IndexIndexes41.559.6+18.1
VoxelBench Text RatingGames22972550+18.0
ExploitBench - Internal Port (June-August 2026)Cybersecurity3.521.5+18.0
HPCT Refusal (CAIS)Safety24.042.0+18.0
OpenAI internal computer-use safety - adverse outcome rateSafety22.04.3+17.7
Terminal-Bench 4.0Code37.955.1+17.2
FrontierMath Tier 4 (v2) - PrivateMath82.9100.0+17.1
BullshitBench v2 LeaderboardSafety48.065.0+17.0
LMArena Code Arena WebDevCode16151743+16.7
Humanity's Sixth SenseMultimodal30.046.6+16.6
Terminal-Bench 4.0 (Artificial Analysis)Code39.956.1+16.2
ProofBench v1.1Math83.099.0+16.0
TxBench Oligo Discovery V1 (113 evaluations)Science37.252.5+15.3
RuneBenchGames5.96.9+15.0
LMArena Agent Arena Net Improvement (Steering Burden)Agents4.612.5+14.8
MetagenomicsBench V1Science31.345.7+14.3
RareDiseaseBench V1 (35 evaluations)Science13.327.6+14.3
MysteryMechanismScience33.346.4+13.1
LisanBench Path LengthGames46998736+12.5
FrontierPhysics v0.1Science7.119.6+12.5
DAYJOB: FinanceFinance & Accounting5.518.0+12.5
TxBench Antibody Discovery V1 (100 evaluations)Science37.349.7+12.3
MindCube TinyMultimodal76.889.1+12.3
Code MigrationCode52.965.1+12.2
GameBench 2Games80.192.0+11.9
Machiavelli Deception (CAIS)Safety82.778.8+11.8
GBENCH One-shot Coding GScoreGames67.779.0+11.3
MentalHealthBenchSafety46.757.9+11.2
FrontierCode 2.0 - Pass RateCode32.343.0+10.8
FrontierCode 2.0 - ScoreCode28.939.6+10.8
VariantBench V1 Clean RerunsScience35.346.0+10.7
LMArena Agent Arena Praise vs ComplaintAgents6.327.5+10.6
BioSecBench-Function V1 Clean RerunsScience35.546.0+10.5
AA-Omniscience IndexIndexes22.041.5+9.8
Olam Social Poker EloGames15341543+9.8
AutomationBench 1.0.6 (provider-reported)Agents45.836.1−9.7
Legal Research BenchLegal48.138.5−9.6
SpatialViz-BenchMultimodal80.790.3+9.6
EBR-bench v4 (card ban, single agent)Games44.854.3+9.5
Roboflow Vision Evals (October 2026, five tasks)Multimodal72.481.7+9.3
LiveBench 2.0Agents76.385.4+9.1
τ³-Banking (Fireworks, banking_knowledge)Agents35.744.7+9.0
WellnessBenchSafety57.066.0+9.0
Chess Puzzles (task version 2.0)Games59.068.0+9.0
GeneBench-Pro Public Release SubsetScience24.033.0+9.0
TxBench-Antibody-Development V1 (60 evaluations)Science18.927.8+8.9
BioMysteryBench (Vals Terminus 2)Science71.179.6+8.5
Vibe Code Bench v1.1Code80.588.9+8.4
IntPhys 2Multimodal63.972.3+8.4
Terminal-Bench 2.1 Extended (Mercor held-out)Code37.445.8+8.4
TaxCalcBench TY25Finance & Accounting30.038.0+8.0
SM BenchSafety61.169.0+7.9
BigFinanceBench private holdout (final-answer accuracy)Finance & Accounting44.252.0+7.8
PostTrainBench v1.2Indexes31.038.8+7.8
Medical Long Context Reasoning (MLCR-AA)Health & Medicine26.133.9+7.8
YC-BenchGames728001936267+7.3
Public Benefits Bench v1.1Professional Work66.559.3−7.2
ERQAMultimodal71.278.2+7.0
TextQuests Average Cumulative HarmSafety31.429.3+6.9
SUPERChemScience71.077.8+6.8
BioSecBench-RespiratoryVirus-Surveillance V1 (110 evaluations)Science5.812.4+6.7
KernelBench Mega Normalized (RTX PRO 6000)Code7.414.1+6.6
Roboflow Vision Evals (September 2026 protocol)Multimodal79.085.5+6.5
AutomationBench-AA 1.0.6Agents60.166.6+6.5
Financial Coaching BenchFinance & Accounting74.968.5−6.3
GameDevBenchGames63.770.0+6.3
NeuroscienceBench V1 (91 evaluations)Science15.822.0+6.2
NoCoT-Bench NCKI (kspine v3)General QA121130+6.2
ClockBench AccuracyMultimodal68.374.4+6.1
SAGEMath52.646.5−6.0
DAYJOB: HealthcareHealth & Medicine1.67.6+6.0
Artificial Analysis Progress Index v1Indexes331348+6.0
ArXivMath 06/2026Math88.594.4+5.9
HardGeoBench Avg DistanceMultimodal14291239+5.9
Humanity's Last ExamGeneral QA45.551.3+5.8
IOI (Vals v2)Code91.296.9+5.7
OSWorld 2 (Aug 8) offline - reported partial reward, step limit unspecifiedAgents65.871.4+5.7
Tax Agent BenchFinance & Accounting68.062.3−5.6
Artificial Analysis Coding Agent Index v1.5Indexes54.660.1+5.6
Conceptual Reasoning IndexIndexes68.772.7+5.4
CritPt v2 (agentic, avg@5)Science77.182.6+5.4
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine18.724.0+5.3
SWE Atlas - Codebase QnACode53.548.4−5.1
HardGeoBench <=10miMultimodal30.035.0+5.0
HardGeoBench Country AccuracyMultimodal47.552.5+5.0
Toloka Arena (v3)Agents61.066.0+5.0
Agents' Last Exam Overall ScoreAgents53.658.6+5.0
ACCoRDGeneral QA75.980.9+5.0
Finance Agent v2 - Disclosure AnalysisFinance & Accounting63.358.4−4.9
MedCodeHealth & Medicine44.048.8+4.9
Artificial Analysis Intelligence Index v4.3.2Indexes47.051.8+4.9
VCT-Refusal (CAIS)Safety11.316.1+4.8
BrowseComp (Mercor 130-question subset)Search90.695.4+4.8
ExploitGym - release-reported percentage, cohort unspecifiedCybersecurity30.335.1+4.8
Epoch Capabilities Index (ECI)Indexes163169+4.7
APEX-Agents 1.1 Corporate Law (68-task image-free slice)Legal80.685.3+4.7
LMArena Agent Arena Confirmed SuccessAgents2.311.7+4.7
ChessBench AccuracyGames76.480.7+4.6
FrontierMath Tiers 1-3 (v2) - PrivateMath89.193.7+4.6
Finance Agent v2 - AdjustmentsFinance & Accounting49.244.7−4.5
FrontierMath ErdősMath0.04.4+4.4
PCT Political Inconsistency (CAIS)Safety41.737.3+4.4
GDP.pdfProfessional Work30.735.0+4.3
LMCAGeneral QA58.562.8+4.3
DeepsecBench — Revalidated V1 (233 issues)Code35.339.5+4.2
AA-Briefcase v1.1 EloAgents14791555+4.2
Terminal-Bench 2.1Code89.585.4−4.1
EnterpriseBench: CoreCraftProfessional Work46.250.3+4.1
RiemannBench (November 2026 rescoring)Math83.279.2−4.0
LMArena Agent Arena Steering BurdenAgents5.513.4+3.9
RCM-Bench-1Health & Medicine91.287.4−3.8
HieroglyphBenchMultimodal27.631.4+3.8
Arena Alignment IndexSafety84.287.9+3.8
ScBench V1 Clean RerunsScience59.863.6+3.8
BioSecBench-Surveillance V1 Clean RerunsScience47.350.8+3.5
ObviousBenchGeneral QA96.5100.0+3.5
LMArena Agent Arena Bash ReliabilityAgents11.45.0−3.2
Vals Index v2.1Indexes58.061.2+3.1
CritPt (avg@5)Science28.631.7+3.1
HealthBench ProfessionalHealth & Medicine64.167.2+3.1
HealthBench Hard (length-adjusted)Health & Medicine33.136.2+3.1
Harvey's Legal Agent BenchmarkLegal2.55.4+2.9
MASKSafety95.498.3+2.9
Finance Agent v2 - Earnings AnalysisFinance & Accounting74.471.5−2.9
ScBench-Long V1 Clean RerunsScience29.626.7−2.8
Bedside BenchHealth & Medicine91.193.9+2.8
Vals RSI Index v1.1 (four tasks)Agents23.926.6+2.7
Harvey LAB: In-House Contracting (held-out)Legal2.04.7+2.7
Finance Agent v2 - ComparablesFinance & Accounting45.342.6−2.7
Longtail-SWECode75.778.4+2.6
LLM Chess EloGames15501612+2.5
DeepSWE v1.1Code72.775.2+2.5
EpiBench V1 Clean RerunsScience25.828.3+2.5
AA-AnalystAgent (pass^5)Professional Work47.550.0+2.5
HardGeoBench <=100miMultimodal41.343.8+2.5
MMMU ProMultimodal82.785.1+2.4
Finance Agent v2 - Financial ModelingFinance & Accounting20.723.1+2.4
SpatialBench-Long V1 Clean RerunsScience37.539.9+2.4
HealthBench Hard (updated implementation)Health & Medicine31.133.4+2.3
SciCode 1.0.1 (Artificial Analysis)Code57.855.8−2.0
LMArena Agent Arena Bash RecoveryAgents0.74.5+1.9
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting64.963.2−1.7
Extended NYT Connections 940 (Quadratic v1)Games93.895.5+1.7
SpeciEval (Refusal-Aware Retries, Historical)Safety99.097.4−1.7
ARC-AGI-2General QA92.594.2+1.7
SpeciEval (Pooled English Runs)Safety99.097.4−1.7
TxBench Preclinical Pharmacology V1 Clean RerunsScience61.359.7−1.7
HealthBenchHealth & Medicine55.356.9+1.6
SciCode (main problems)Science16.214.6−1.6
ChessBench CoherenceGames89.187.5−1.6
Excel Modeling BenchmarkFinance & Accounting72.370.8−1.6
ProgramBenchCode1.53.0+1.5
Finance Agent v2 - All PassFinance & Accounting41.439.9−1.5
MedScribeHealth & Medicine85.286.5+1.2
VulcanBench Frontier v4Code87.288.3+1.2
Vision Arena OverallMultimodal12881293+1.2
APEX-SWECode48.046.9−1.1
MacroscopeBenchCode77.876.7−1.1
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting75.674.5−1.1
ARC-AGI-1 Semi-PrivateGeneral QA97.598.5+1.0
SpatialBench V1 Clean RerunsScience68.167.3−0.9
DTBenchGeneral QA95.796.5+0.8
Wiz Cyber Model Arena v3.9Cybersecurity69.870.6+0.8
Bug Hunt BenchCode41.442.2+0.8
FrontierCode 1.1 Extended - Pass RateCode66.665.9−0.7
FrontierCode 1.1 Extended - ScoreCode60.559.8−0.7
DistressBenchSafety73.674.2+0.6
SWE-Bench Pro V2 — FullCode95.596.1+0.6
BioSecBench-Structure V1 (58 evaluations)Science20.721.3+0.6
LMArena Text Arena EloChat & Writing14851482−0.5
SWE-TogetherCode57.858.3+0.5
GDPval-AA v2.1Professional Work16191607−0.5
Finance Agent v2 - PrecedentsFinance & Accounting26.225.8−0.4
Finance Agent v2 - Market AnalysisFinance & Accounting64.264.5+0.3
ForecastBench Preliminary Dataset Brier IndexForecasting61.461.7+0.3
CVDP (517-problem set)Code55.455.1−0.3
AIM RoutingAgents89.189.3+0.2
Ruby LLM Benchmarks — Program FixerCode75.275.0−0.1
Prophet Arena Arena Score (Fixed Context)Forecasting84.884.80.0
LMArena Agent Arena Tool HallucinationAgents1.01.00.0
AA-LCR v1.1Search84.084.00.0
OTIS Mock AIME 2024-2025Math100.0100.00.0
SWE-Marathon v1.1Code42.542.50.0

GPT-5.6 SolvsGPT-6.1 Solnormalised score marginsData Nov 15, 2026Data: Vals AI, benchmarks.bio, Artificial Analysis and 65 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…