Atlas

Compare Models

GPT-5.6 Sol vs Grok 4.6

GPT-5.6 Sol leads by 2.9 capability points (≈95% interval +0.8 to +4.9): a clear lead over Grok 4.6 · P(higher) ≈ 100%

Wins 170 of 285 shared benchmarks (98 lost, 17 tied)

Capability vs. relative cost
GPT-5.6 SolGrok 4.6Closed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

GPT-5.6 SolvsGrok 4.6capability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins170 – 98 · 285 shared, 17 tied

Comparing matching setups only (90 benchmarks excluded) · suites counted once
← GPT-5.6 SolGrok 4.6 →

GPT-5.6 SolvsGrok 4.6wins by categoryData Nov 15, 2026Data: Vals AI, benchmarks.bio, Artificial Analysis and 75 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Grok 4.6 minus GPT-5.6 Sol.

RT-SAFE Collisions per EpisodeSafety25.659.5−85.0
LLM Creative Story WritingChat & Writing2.5-2.9−60.5
FrontierMath Tier 4 (v2) - PrivateMath82.931.7−51.2
LLM Chess EloGames1550443−45.4
IOI (Vals v2)Code91.247.6−43.6
MRCR v2 (8-needle, Context Arena) - 128K pointwiseGeneral QA92.452.6−39.8
Olam Social Poker EloGames15341499−38.0
Document ArenaProfessional Work14801438−37.5
ProofBench v1.1Math83.051.0−32.0
YC-BenchGames7280011568241+29.3
KernelBench Hard Normalized (RTX PRO 6000)Code31.960.5+28.5
SM BenchSafety61.189.0+27.9
Chess Puzzles (task version 2.0)Games59.033.0−26.0
ARC-AGI-2General QA92.567.1−25.4
Long-Horizon Terminal-Bench 2.0 (mean reward)Code67.242.0−25.2
Design Creativity BenchCode51.343.4−25.1
Gray Swan IPI Attack Success (k=15)Safety27.051.8−24.8
BioSecBench-RefusalSafety39.564.2+24.7
ChartographyMultimodal45.020.5−24.5
Mystery Game PuzzlesGames58.034.0−24.0
LAB-Bench LitQA2 (SecureBio)Science52.828.9−23.9
SAGEMath52.628.9−23.7
Bio Capabilities Index (SecureBio)Indexes169137−23.3
FrontierMath Tiers 1-3 (v2) - PrivateMath89.166.0−23.2
MRCR v2 (8-needle, Context Arena) - 128K AUCGeneral QA97.174.4−22.7
Concept Synth INDUCTION Challenge64 Round 1General QA57.835.9−21.9
Antidote: Everyday EditionSafety1027966−21.8
Harvey LAB held-out set (Fireworks run)Legal0.622.1+21.5
WeirdML (v2)Science88.867.3−21.5
MirrorCode (ML, +Private, 2L)Code20.041.1+21.1
ALE-BenchCode24101822−20.8
Hemingway-benchChat & Writing1057983−20.5
Wiz Cyber Model Arena v3.9Cybersecurity66.946.6−20.3
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal20.040.0+20.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.020.0+20.0
ChessBench CoherenceGames89.169.2−19.9
ScBench-Long V0 VerifiedScience43.924.2−19.7
ChessBench EloGames1218891−19.5
EnterpriseBench: CoreCraftProfessional Work46.265.6+19.4
DeepsecBench — Revalidated V1Code35.416.2−19.3
DeepsecBench — Revalidated V1 (233 issues)Code35.316.1−19.2
TxBench Antibody Discovery V0Science33.851.9+18.1
Concept Synth INDUCTION Challenge100 Round 1General QA43.025.0−18.0
Time Horizon Index: KSPAgents23.85.8−18.0
VoxelBench Text RatingGames22972050−17.6
LMArena Agent Arena Net ImprovementAgents6.31.1−16.9
SciUniverse Level 1Science9.426.2+16.8
Furniture Assembly BenchmarkMultimodal56.740.0−16.7
SpatialBench-Long V0 VerifiedScience36.419.7−16.7
Terminal-Bench-Science 0.1Science22.46.2−16.2
HealthBench Professional (Length-Adjusted)Health & Medicine60.544.4−16.1
CadQueryEval (Corrected Geometry)Code88.072.0−16.0
BuseyBench SVGMultimodal6.14.6−15.0
CritPt v2 (agentic, avg@5)Science77.162.3−14.9
HPCT (SecureBio)Science76.361.6−14.7
WCB (SecureBio)Science64.450.3−14.1
ComplexConstraintsChat & Writing50.536.5−14.0
WellnessBenchSafety57.043.0−14.0
Medical Long Context Reasoning (MLCR-AA)Health & Medicine26.112.2−13.9
LAB-Bench ProtocolQA (SecureBio)Science81.968.1−13.8
Extended NYT Connections 940 (Quadratic v1)Games93.880.0−13.8
GDP.pdfProfessional Work30.717.2−13.5
Roboflow Vision Evals (October 2026, five tasks)Multimodal72.459.1−13.3
TxBench Antibody Discovery V1 Clean RerunsScience31.444.5+13.1
ScBench-Long V1 Clean RerunsScience29.616.5−13.1
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting64.977.8+12.8
ReactBench (39 tasks)Code46.733.8−12.8
NoCoT-Bench NCRI 15.2General QA10986.8−12.5
GBENCH One-shot Coding GScoreGames67.755.4−12.3
Veracode GenAI Code Security (Fall 2026)Cybersecurity70.058.0−12.0
Terminal-Bench 4.0Code37.325.3−12.0
Harvey HLAB — Criteria Pass RateLegal80.592.5+12.0
APEX-SWECode45.857.7+11.9
HardGeoBench Avg DistanceMultimodal14291807−11.6
GameBench 2Games80.168.7−11.3
SpatialBench-Long V1 Clean RerunsScience37.526.2−11.3
ZeroBench Main Mean pass@1 (five samples)Multimodal22.011.0−11.0
BrowseComp Extended (Mercor held-out)Search32.321.5−10.8
HackerRank-BenchCode38.427.7−10.8
MBCT (SecureBio)Science61.150.4−10.7
Roboflow Vision Evals (September 2026 protocol)Multimodal79.068.7−10.3
LMCAGeneral QA58.548.5−10.0
ARC-AGI-1 Semi-PrivateGeneral QA97.587.5−10.0
Co-Design BenchCode61.151.1−10.0
HardGeoBench <=10miMultimodal30.020.0−10.0
Excel Modeling BenchmarkFinance & Accounting72.362.7−9.6
TxBench Oligo Discovery V0Science34.544.0+9.4
PWNBench v0.1Cybersecurity54.063.3+9.3
ChessBench AccuracyGames76.467.8−9.2
LMArena Agent Arena Net Improvement (Steering Burden)Agents4.6-0.2−9.1
GeneBench-Pro Public Release SubsetScience24.015.0−9.0
Humanity's Last Exam DiamondGeneral QA30.421.5−8.9
CritPt (avg@5)Science28.619.7−8.9
HardGeoBench <=100miMultimodal41.332.5−8.7
CyberBench v1.1 OverallCybersecurity76.367.7−8.6
Terminal-Bench Science 0.1 (Vals)Science20.011.4−8.6
RareDiseaseBench V1 (35 evaluations)Science13.34.8−8.6
Conceptual Reasoning IndexIndexes68.762.5−8.6
BioSecBench-Function V1 Clean RerunsScience35.544.0+8.5
Senior SWE-BenchCode34.726.3−8.4
Terminal-Bench 3 - release-reported accuracyCode34.426.0−8.4
Code MigrationCode52.944.6−8.3
TxBench Oligo Discovery V1 Clean RerunsScience35.043.3+8.3
UnityBench-Mini3DCode62.370.6+8.3
SpeciEval (Refusal-Aware Retries, Historical)Safety99.090.8−8.2
SpeciEval (Pooled English Runs)Safety99.090.8−8.2
Terminal-Bench 2.1 Extended (Mercor held-out)Code37.445.5+8.1
FoodTruck BenchGames5322959537+8.1
Eval Connections One-Shot with TrapsGames83.091.0+8.0
Harvey LAB: In-House Contracting (held-out)Legal2.010.0+8.0
WordleBenchGames97.089.0−8.0
EpiBenchScience42.134.3−7.9
Bedside BenchHealth & Medicine91.183.3−7.8
KernelBench Mega Normalized (RTX PRO 6000)Code7.40.0−7.4
MetagenomicsBench V0Science31.338.7+7.4
MetagenomicsBench V1Science31.338.7+7.3
DAYJOB: FinanceFinance & Accounting5.512.8+7.3
HieroglyphBenchMultimodal27.620.3−7.3
TxBench-Antibody-Development V1 (60 evaluations)Science18.926.1+7.2
SCI-AGENT-EXAMScience59.652.4−7.2
MA-BenchProfessional Work51.258.3+7.1
FrontierSWE V2Code32.225.3−6.9
AutomationBench-AA 1.0.6Agents60.167.0+6.9
BrowseComp (Mercor 130-question subset)Search90.684.0−6.6
AA-Briefcase EloAgents14351545+6.4
𝜏³-BankingAgents44.350.7+6.4
Vision Arena OverallMultimodal12881263−6.4
AA-AnalystAgent (pass^5)Professional Work47.541.3−6.2
NoCoT-Bench NCKI (kspine v3)General QA121112−6.2
VIALSMultimodal26.520.5−6.0
ZeroBench Main Questions pass^5Multimodal13.07.0−6.0
VCT (SecureBio)Science54.348.3−6.0
Vals Index v2.1Indexes58.052.1−5.9
LLM Debate BenchmarkChat & Writing16451604−5.8
BioSecBench-RespiratoryVirus-Surveillance V1 (110 evaluations)Science5.811.5+5.8
LiveCodeBench (Vals AI public dataset)Code82.688.2+5.6
EnterpriseOps-Gym-AAAgents42.948.3+5.4
SUPERChemScience71.065.7−5.3
BioSecBench-Structure V1 (58 evaluations)Science20.715.5−5.2
SkillsBench without SkillsAgents43.338.2−5.1
VariantBench V1 Clean RerunsScience35.340.4+5.1
Hedge-BenchFinance & Accounting31.526.4−5.1
Toloka Arena (v3)Agents61.056.0−5.0
LMArena Text Arena EloChat & Writing14851456−5.0
ARC-AGI-3 (Semi-Private)General QA7.02.1−4.9
Humanity's Last ExamGeneral QA44.539.7−4.8
NeuroscienceBench V1 (91 evaluations)Science15.820.5+4.8
CheatBenchSafety74.979.6−4.7
SpatialBench V0 VerifiedScience74.569.9−4.6
LAB-Bench CloningScenarios (SecureBio)Science98.894.2−4.6
Vals Index v2Indexes63.759.2−4.5
GlobeBenchGeneral QA23.728.1+4.5
LMArena Agent Arena Praise vs ComplaintAgents6.3-2.5−4.4
ACCoRDGeneral QA75.971.6−4.3
Finance Agent v2 - PrecedentsFinance & Accounting26.221.9−4.3
AA-Omniscience IndexIndexes22.030.5+4.3
Vibe Code Bench v1.1Code80.576.2−4.3
Epoch Capabilities Index (ECI)Indexes163158−4.1
Finance Agent v2 - ComparablesFinance & Accounting45.341.2−4.1
BabaIsBench Stage 0: The IntroGames100.096.0−4.0
VariantBenchScience42.138.1−4.0
DeepSWE v1.1Code70.766.7−4.0
GDPval-AA v2Professional Work15851663+3.8
PG-LLM-50Science0.40.3−3.8
HANDBOOK.mdProfessional Work23.527.3+3.8
HardGeoBench Country AccuracyMultimodal47.543.8−3.7
BioSecBench-Function V0Science38.342.0+3.7
TaxEval v2Finance & Accounting74.871.1−3.7
LMArena Agent Arena Steering BurdenAgents5.5-1.8−3.7
GDPval-AA v2.1Professional Work15771658+3.6
Artificial Analysis Progress Index v1Indexes331322−3.5
ObviousBenchGeneral QA96.5100.0+3.5
Vending-Bench 2Games96199047−3.5
Tax Agent BenchFinance & Accounting68.071.4+3.4
Arena Alignment IndexSafety84.280.8−3.4
Vals Science IndexIndexes41.538.1−3.4
SWE Atlas - Codebase QnACode53.550.3−3.2
MortgageTax (Vals all 1,258 documents)Finance & Accounting67.364.2−3.1
AA-LCR v1.1Search84.081.0−3.0
CrossSearch (pass^3)Search38.035.0−3.0
CorpFin v2 - Exact PagesFinance & Accounting61.564.5+2.9
SWE-TogetherCode57.860.6+2.8
AHK-EvalCode94.497.2+2.8
MMLU Pro - LawGeneral QA79.181.8+2.7
MysteryMechanismScience33.330.6−2.7
CursorBench 3.2Code67.269.9+2.7
MacroscopeBenchCode77.875.1−2.7
Artificial Analysis Intelligence Index v4.3.2Indexes47.044.3−2.7
Artificial Analysis Intelligence Index v4.3Indexes47.144.4−2.7
CVDP (517-problem set)Code55.458.0+2.6
ORCA-bench Hard RCA (Fireworks 324-task subset)Agents29.532.1+2.5
LMArena Agent Arena Bash RecoveryAgents0.75.7+2.5
LMArena Agent Arena Confirmed SuccessAgents2.3-2.6−2.4
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting75.673.2−2.4
EpiBench V1 Clean RerunsScience25.823.6−2.2
MMMU ProMultimodal82.780.5−2.2
GPQA-Bio (SecureBio)Science85.387.4+2.1
Toloka Arena (v1)Agents47.649.7+2.1
MMLU Pro - EngineeringGeneral QA87.885.8−2.1
AutoResearchExam (hidden-test AUARC)Agents0.50.5−2.0
Shroombench (mushroom identification, top-1)Multimodal42.044.0+2.0
Finance Agent v2 - Market AnalysisFinance & Accounting64.262.2−1.9
Finance Agent v2 - Financial ModelingFinance & Accounting20.718.9−1.8
RedlineBenchLegal50.348.5−1.8
SkillsBench with SkillsAgents54.155.8+1.7
LMArena Agent Arena SteerabilityAgents6.53.3−1.6
MMLU Pro - PsychologyGeneral QA89.290.9+1.6
SciCode (main problems)Science16.214.6−1.6
APEX-Agents 1.1 Corporate Law (68-task image-free slice)Legal80.682.2+1.6
DTBenchGeneral QA95.797.3+1.6
Finance Agent v2 - All PassFinance & Accounting41.442.9+1.5
Terminal-Bench 2.1Code89.588.0−1.5
CorpFin v2 - Shared Max ContextFinance & Accounting66.467.8+1.4
LiveBench 2.0Agents76.374.9−1.3
MedScribeHealth & Medicine85.286.5+1.3
SciCode 1.0.1 (Artificial Analysis)Code57.856.5−1.3
DAYJOB: HealthcareHealth & Medicine1.62.8+1.2
MMLU Pro - EconomicsGeneral QA90.691.8+1.2
BioMysteryBench (Vals Terminus 2)Science71.172.2+1.1
CorpFin v2 - Max Fitting ContextFinance & Accounting65.266.2+1.0
Finance Agent v2 - Disclosure AnalysisFinance & Accounting63.364.4+1.0
MMLU Pro - Computer ScienceGeneral QA90.791.7+1.0
SAB ResearchFinance & Accounting93.694.6+1.0
SpatialBench V1 Clean RerunsScience68.167.3−0.9
OTIS Mock AIME 2024-2025Math100.099.2−0.8
MMLU Pro - PhilosophyGeneral QA88.489.2+0.8
BioSecBench-SurveillanceScience52.551.7−0.8
MMLU Pro - HistoryGeneral QA81.981.1−0.8
MedCodeHealth & Medicine44.044.7+0.7
FrontierCode 1.1 Extended - ScoreCode60.661.3+0.7
MMLU-Bio (SecureBio)Science91.191.8+0.7
LMArena Agent Arena Bash ReliabilityAgents11.410.0−0.7
TissueMicroscopyBench V1 (49 evaluations)Science2.02.7+0.7
ScBench V1 Clean RerunsScience59.859.1−0.7
ATLAS-Finance v1.1Finance & Accounting4.04.7+0.7
MMLU Pro - MathMath95.395.9+0.7
LegalBenchLegal87.086.3−0.7
MMLU Pro - BusinessGeneral QA93.392.6−0.6
MMLU Pro - PhysicsGeneral QA92.892.1−0.6
SWE-bench VerifiedCode96.295.6−0.6
GPQA DiamondGeneral QA94.995.5+0.5
Terminal-Bench 4.0 (Artificial Analysis)Code20.721.2+0.5
ProgramBenchCode1.51.0−0.5
Finance Agent v2 - Earnings AnalysisFinance & Accounting74.473.9−0.5
MMLU Pro - OtherGeneral QA87.787.2−0.4
MMLU Pro - BiologyGeneral QA94.695.0+0.4
Blueprint-Bench 2Multimodal0.30.3−0.4
ForecastBench Preliminary Dataset Brier IndexForecasting61.461.0−0.4
Finance Agent v2 - AdjustmentsFinance & Accounting49.249.6+0.4
TxBench Preclinical Pharmacology V1 Clean RerunsScience61.361.7+0.3
Public Benefits Bench v1.1Professional Work66.566.8+0.3
MMLU Pro (Vals 5-Shot CoT)General QA89.189.4+0.3
CursorBench 4.0Code41.741.4−0.3
ForecastBench Tournament Overall Brier IndexForecasting66.165.8−0.3
BioSecBench-Surveillance V1 Clean RerunsScience47.347.5+0.3
MMLU Pro - HealthGeneral QA84.184.4+0.2
LibraryDesignBenchCode39.539.7+0.2
Realm Financial ReasoningFinance & Accounting44.444.6+0.2
Prophet Arena Arena Score (Fixed Context)Forecasting84.885.0+0.1
MMLU Pro - ChemistryGeneral QA92.092.0+0.1
LMArena Agent Arena Tool HallucinationAgents1.01.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
Legal Research BenchLegal48.148.10.0
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine18.718.70.0

GPT-5.6 SolvsGrok 4.6normalised score marginsData Nov 15, 2026Data: Vals AI, benchmarks.bio, Artificial Analysis and 75 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…