Atlas

Compare Models

GPT-5.6 Sol vs GPT-6 Sol

GPT-6 Sol leads by 0.9 capability points (≈95% interval −1.4 to +3.2): statistically tied · P(higher) ≈ 78%

Wins 127 of 230 shared benchmarks (99 lost, 4 tied)

Capability vs. relative cost
GPT-5.6 SolGPT-6 SolClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

GPT-5.6 SolvsGPT-6 Solcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins99 – 127 · 230 shared, 4 tied

Comparing matching setups only (14 benchmarks excluded) · suites counted once
← GPT-5.6 SolGPT-6 Sol →

GPT-5.6 SolvsGPT-6 Solwins by categoryData Nov 15, 2026Data: Vals AI, Artificial Analysis, benchmarks.bio and 70 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6 Sol minus GPT-5.6 Sol.

Search ArenaSearch12581174−66.8
BrowseComp Extended (Mercor held-out)Search32.363.2+30.9
Vending-Bench 2Games961914428+29.1
BrokenArXiv 06/2026Math67.395.4+28.1
YC-BenchGames7280011441793+24.9
SecPRBench (recall)Cybersecurity55.876.2+20.4
Legal Research BenchLegal48.128.8−19.2
LisanBench Difficulty-WeightedGames14033118+18.4
Gray Swan IPI Attack Success (k=15)Safety27.010.1+16.9
Tax Agent BenchFinance & Accounting68.053.0−14.9
Humanity's Last Exam DiamondGeneral QA30.445.2+14.8
BuseyBench SVGMultimodal6.17.5+14.2
Bug Hunt BenchCode41.427.9−13.5
AURORA Multilingual τ³-bench (Persian)Agents56.468.9+12.5
Financial Coaching BenchFinance & Accounting74.962.6−12.3
CadQueryEval (Corrected Geometry)Code88.0100.0+12.0
τ³-Banking (Fireworks, banking_knowledge)Agents35.747.0+11.3
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting64.953.6−11.3
LisanBench Path LengthGames46998222+10.9
AURORA Multilingual τ³-bench (Turkish)Agents58.569.3+10.8
Arena Image-to-WebDevCode16061655+10.7
MetagenomicsBench V0Science31.342.0+10.7
MetagenomicsBench V1Science31.342.0+10.7
AURORA Multilingual τ³-bench (English)Agents63.974.1+10.1
Medical Long Context Reasoning (MLCR-AA)Health & Medicine26.116.1−10.0
Terminal-Bench Science 0.1 (Vals)Science20.030.0+10.0
Public Benefits Bench v1.1Professional Work66.556.6−9.9
RCM-Bench-1Health & Medicine91.281.5−9.7
RiemannBenchMath74.464.8−9.6
SWE Atlas - Codebase QnACode53.544.1−9.4
Biopharma Bench V0.1Health & Medicine44.954.3+9.4
HealthBench Hard (updated implementation)Health & Medicine31.122.1−9.0
Olam Social Poker EloGames15341542+8.7
ChartographyMultimodal45.053.6+8.6
DeepResearch Bench II (Mercor 130-task set)Search51.542.9−8.6
EBR-bench v4 (card ban, single agent)Games44.853.3+8.6
IOI (Vals v2)Code91.282.6−8.6
HPCT (SecureBio)Science76.367.8−8.5
RuneBenchGames5.96.4+8.4
LMArena Agent Arena Net ImprovementAgents6.38.8+8.4
ExploitGym - release-reported percentage, cohort unspecifiedCybersecurity30.322.1−8.2
ALE-BenchCode24102637+8.0
AURORA Multilingual τ³-bench (Korean)Agents63.171.0+8.0
SAGEMath52.644.8−7.8
Creative Writing v3 EloChat & Writing19742126+7.7
Finance Agent v2 - Disclosure AnalysisFinance & Accounting63.355.6−7.7
Terminal-Bench-Science 0.1Science22.430.0+7.6
LMArena Code Arena WebDevCode16151673+7.5
MentalHealthBenchSafety46.754.2+7.5
Vibe Code Bench v1.1Code80.587.8+7.3
AURORA Multilingual τ³-bench (German)Agents65.572.8+7.3
FrontierPhysics v0.1Science7.114.3+7.2
FrontierMath Tier 4 (v2) - PrivateMath82.990.0+7.1
HealthBenchHealth & Medicine55.348.2−7.1
VoxelBench Text RatingGames22972394+6.9
BioSecBench-Structure V1 (58 evaluations)Science20.713.8−6.9
ZeroBench Main Mean pass@1 (five samples)Multimodal22.028.8+6.8
ITBench-AAAgents56.249.4−6.8
FORTRESS Average Risk Score (ARS)Safety20.113.5+6.7
Terminal-Bench 4.0Code37.944.4+6.6
ACCoRDGeneral QA75.982.4+6.5
dfbench Detection RecallCybersecurity65.759.3−6.4
Finance Agent v2 - Market AnalysisFinance & Accounting64.258.0−6.2
DeepSWE v1.1Code72.766.6−6.1
FrontierCode 2.0 - Pass RateCode32.338.3+6.1
FrontierCode 2.0 - ScoreCode28.934.9+6.0
BullshitBench v2 LeaderboardSafety48.054.0+6.0
Eval Connections One-Shot with TrapsGames83.089.0+6.0
Finance Agent v2 - AdjustmentsFinance & Accounting49.243.2−6.0
SpatialBench-Long V1 Clean RerunsScience37.531.6−5.9
Vectara Hallucination LeaderboardSafety12.46.5+5.9
LiveBench 2.0Agents76.381.9+5.6
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting75.670.0−5.6
DistressBenchSafety73.668.0−5.6
TxBench Antibody Discovery V1 Clean RerunsScience31.437.0+5.6
TxBench Oligo Discovery V1 Clean RerunsScience35.040.6+5.6
DeepsecBench — Revalidated V1Code35.440.9+5.5
Bio Capabilities Index (SecureBio)Indexes169161−5.5
DeepsecBench — Revalidated V1 (233 issues)Code35.340.7+5.4
Finance Agent v2 - All PassFinance & Accounting41.436.0−5.4
ChessBench EloGames12181303+5.1
CheatBenchSafety77.071.9+5.0
Terminal-Bench 2.1 Extended (Mercor held-out)Code37.442.4+5.0
LMArena Text Arena EloChat & Writing14851457−4.9
SM BenchSafety61.166.0+4.9
KernelBench Mega Normalized (RTX PRO 6000)Code7.42.7−4.7
WeirdML v3Science15.019.7+4.7
GDPval-AA v2.1Professional Work16191514−4.6
OpenAI internal computer-use safety - adverse outcome rateSafety22.017.4+4.6
HealthBench ProfessionalHealth & Medicine64.159.5−4.6
Roboflow Vision Evals (October 2026, five tasks)Multimodal72.476.9+4.5
Finance Agent v2 - ComparablesFinance & Accounting45.340.8−4.5
AutomationBench 1.0.6 (Zapier)Agents28.833.2+4.4
LMArena Agent Arena Net Improvement (Steering Burden)Agents4.66.9+4.3
Code MigrationCode52.957.2+4.3
Vals RSI Index v1.1 (four tasks)Agents23.928.1+4.2
Terminal-Bench 4.0 (Artificial Analysis)Code39.943.9+4.0
LMArena Agent Arena SteerabilityAgents6.514.5+4.0
ZeroBench Main Questions pass^5Multimodal13.017.0+4.0
WCB (SecureBio)Science64.460.5−3.9
Ruby LLM Benchmarks — Program FixerCode75.279.0+3.9
DAYJOB: FinanceFinance & Accounting5.59.3+3.8
EpiBench V1 Clean RerunsScience25.822.0−3.8
BioSecBench-Function V1 Clean RerunsScience35.539.3+3.8
HardGeoBench <=100miMultimodal41.337.5−3.7
HardGeoBench <=10miMultimodal30.026.3−3.7
CritPt v2 (agentic, avg@5)Science77.180.9+3.7
BioMysteryBench (Vals Terminus 2)Science71.174.8+3.7
Extended NYT Connections 940 (Quadratic v1)Games93.890.1−3.7
GPQA-Bio (SecureBio)Science85.388.9+3.6
FrontierCUA pass@1Agents16.420.0+3.6
Vals Science IndexIndexes41.545.0+3.5
MacroscopeBenchCode77.874.3−3.5
SciCode (main problems)Science16.212.7−3.5
Nimble Web ContextSearch80.076.5−3.5
Arena Alignment IndexSafety84.287.6+3.4
VariantBench V1 Clean RerunsScience35.338.7+3.4
Blueprint-Bench 2Multimodal0.30.4+3.3
Roboflow Vision Evals (September 2026 protocol)Multimodal79.082.3+3.3
MedScribeHealth & Medicine85.282.0−3.2
Conceptual Reasoning IndexIndexes68.771.0+3.2
ARC-AGI-3 (Semi-Private)General QA7.84.6−3.2
MysteryMechanismScience33.330.2−3.2
LMArena Agent Arena Bash RecoveryAgents0.77.0+3.2
EnterpriseBench: CoreCraftProfessional Work46.243.1−3.1
MedCodeHealth & Medicine44.047.1+3.1
Toloka Arena (v3)Agents61.058.0−3.0
HealthBench Hard (length-adjusted)Health & Medicine33.130.1−3.0
APEX-SWECode48.045.0−3.0
Bedside BenchHealth & Medicine91.188.1−3.0
Veracode GenAI Code Security (Fall 2026)Cybersecurity70.073.0+3.0
Chess Puzzles (task version 2.0)Games59.056.0−3.0
ARC-AGI-2General QA92.589.6−2.9
BigFinanceBench private holdout (final-answer accuracy)Finance & Accounting44.247.1+2.9
NoCoT-Bench NCRI 15.2General QA109115+2.8
SWE-TogetherCode57.860.6+2.8
ObviousBenchGeneral QA96.599.3+2.8
CVDP (517-problem set)Code55.452.7−2.7
Terminal-Bench 2.1Code85.883.1−2.6
GDP.pdf (Artificial Analysis, All-pass)Professional Work27.825.2−2.6
AA-Omniscience IndexIndexes22.027.1+2.6
ArXivMath 06/2026Math88.591.0+2.4
GBENCH One-shot Coding GScoreGames67.770.1+2.4
Finance Agent v2 - PrecedentsFinance & Accounting26.228.5+2.3
LMArena Agent Arena Steering BurdenAgents5.510.1+2.3
CritPt (avg@5)Science28.630.9+2.3
Artificial Analysis Coding Agent Index v1.5Indexes54.656.7+2.1
HardGeoBench Avg DistanceMultimodal14291497−2.1
DAYJOB: HealthcareHealth & Medicine1.63.6+2.0
ARC-AGI-1 Semi-PrivateGeneral QA97.595.5−2.0
GeneBench-Pro Public Release SubsetScience24.026.0+2.0
ExploitBench - Internal Port (June-August 2026)Cybersecurity3.55.5+2.0
TaxCalcBench TY25Finance & Accounting30.028.0−2.0
Mystery Game PuzzlesGames58.056.0−2.0
Finance Agent v2 - Earnings AnalysisFinance & Accounting74.472.4−2.0
SAB ResearchFinance & Accounting93.695.6+1.9
LMArena Agent Arena Praise vs ComplaintAgents6.310.0+1.8
SUPERChemScience71.072.8+1.8
ScBench V1 Clean RerunsScience59.861.5+1.7
SAB RankingForecasting-0.1-0.1+1.7
TxBench Preclinical Pharmacology V1 Clean RerunsScience61.363.0+1.7
Furniture Assembly BenchmarkMultimodal56.758.3+1.7
CyberBench v1.1 OverallCybersecurity76.378.0+1.7
Agents' Last Exam (ALE)Agents30.632.2+1.6
AutomationBench-AA 1.0.6Agents60.161.7+1.6
HieroglyphBenchMultimodal27.626.0−1.6
DTBenchGeneral QA95.797.3+1.6
Short Form Bench v1.0Games24102394−1.6
FrontierMath ErdősMath0.01.5+1.5
Finance Agent v2 - Financial ModelingFinance & Accounting20.719.3−1.4
Wiz Cyber Model Arena v3.9Cybersecurity69.871.2+1.4
GlobeBenchGeneral QA23.725.0+1.3
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine18.720.0+1.3
OSWorld 2 (Aug 8) offline - reported partial reward, step limit unspecifiedAgents65.864.4−1.3
NoCoT-Bench NCKI (kspine v3)General QA121119−1.3
HardGeoBench Country AccuracyMultimodal47.546.3−1.2
ForecastBench Preliminary Dataset Brier IndexForecasting61.462.6+1.2
Humanity's Sixth SenseMultimodal30.031.2+1.2
BrowseComp (Mercor 130-question subset)Search90.689.4−1.2
ScBench-Long V1 Clean RerunsScience29.628.4−1.1
Vals Index v2Indexes63.762.6−1.1
LMArena Agent Arena Confirmed SuccessAgents2.34.5+1.1
Longform Creative WritingChat & Writing81.782.8+1.1
ChessBench AccuracyGames76.477.4+1.1
Humanity's Last Exam (Text-Only)General QA47.346.3−1.0
LAB-Bench LitQA2 (SecureBio)Science52.851.8−1.0
LibraryDesignBenchCode39.538.5−1.0
WellnessBenchSafety57.056.0−1.0
Artificial Analysis Progress Index v1Indexes331334+0.9
ChessBench CoherenceGames89.188.2−0.9
Harvey's Legal Agent BenchmarkLegal2.51.7−0.8
Excel Modeling BenchmarkFinance & Accounting72.371.5−0.8
LAB-Bench ProtocolQA (SecureBio)Science81.981.1−0.8
Epoch Capabilities Index (ECI)Indexes163164+0.8
AURORA Multilingual Terminal-BenchCode64.565.3+0.7
FrontierMath Tiers 1-3 (v2) - PrivateMath89.189.8+0.7
Harvey LAB: In-House Contracting (held-out)Legal2.01.3−0.7
Artificial Analysis Intelligence Index v4.3.2Indexes47.047.6+0.7
SWE-Marathon v1.1Code42.543.1+0.6
GameBench 2Games80.179.4−0.6
Creative Writing v3 Rubric ScoreChat & Writing16.216.1−0.6
LMCAGeneral QA58.559.1+0.6
SpatialBench V1 Clean RerunsScience68.167.5−0.6
SpeciEval (Refusal-Aware Retries, Historical)Safety99.098.5−0.6
SpeciEval (Pooled English Runs)Safety99.098.5−0.6
ProgramBenchCode1.52.0+0.5
BioSecBench-Surveillance V1 Clean RerunsScience47.347.7+0.5
Vals Index v2.1Indexes58.057.5−0.5
VulcanBench Frontier v4Code87.286.8−0.4
AA-LCR v1.1Search84.083.7−0.3
ComplexConstraintsChat & Writing50.550.8+0.3
LAB-Bench CloningScenarios (SecureBio)Science98.898.5−0.3
MMLU-Bio (SecureBio)Science91.190.8−0.3
WMDP-Bio (SecureBio)Science89.790.0+0.3
VCT (SecureBio)Science54.354.0−0.3
FrontierCode 1.1 Extended - Pass RateCode66.666.3−0.3
LMArena Agent Arena Bash ReliabilityAgents11.410.9−0.2
PubMedQA (SecureBio)Health & Medicine81.180.9−0.2
LLM Chess EloGames15501545−0.2
AA-Briefcase v1.1 EloAgents14791476−0.2
FrontierCode 1.1 Extended - ScoreCode60.560.7+0.1
Harvey LAB held-out set (Fireworks run)Legal0.60.7+0.1
SciCode 1.0.1 (Artificial Analysis)Code57.857.6−0.1
MMMU ProMultimodal82.782.5−0.1
VCT v2 (SecureBio)Science49.449.3−0.1
LMArena Agent Arena Tool HallucinationAgents1.01.00.0
MBCT (SecureBio)Science61.161.10.0
OTIS Mock AIME 2024-2025Math100.0100.00.0
ProofBench v1.1Math83.083.00.0
SRE Bench — Fully SolvedCode30.530.50.0

GPT-5.6 SolvsGPT-6 Solnormalised score marginsData Nov 15, 2026Data: Vals AI, Artificial Analysis, benchmarks.bio and 70 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…