Atlas

Compare Models

GPT-5.6 Sol vs Grok 4.7

GPT-5.6 Sol leads by 2.3 capability points (≈95% interval +0.1 to +4.4): a clear lead over Grok 4.7 · P(higher) ≈ 98%

Wins 133 of 204 shared benchmarks (50 lost, 21 tied)

Capability vs. relative cost
GPT-5.6 SolGrok 4.7Closed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

GPT-5.6 SolvsGrok 4.7capability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins133 – 50 · 204 shared, 21 tied

Comparing matching setups only (86 benchmarks excluded) · suites counted once
← GPT-5.6 SolGrok 4.7 →

GPT-5.6 SolvsGrok 4.7wins by categoryData Nov 15, 2026Data: Vals AI, Artificial Analysis, SecureBio and 57 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Grok 4.7 minus GPT-5.6 Sol.

FrontierMath Tier 4 (v2) - PrivateMath82.917.1−65.9
ProofBench v1.1Math83.026.0−57.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.040.0+40.0
FrontierMath Tiers 1-3 (v2) - PrivateMath89.153.0−36.1
LAB-Bench LitQA2 (SecureBio)Science52.816.9−35.9
Furniture Assembly BenchmarkMultimodal56.720.8−35.8
Design Creativity BenchCode51.362.2+34.7
IOI (Vals v2)Code91.257.7−33.4
YC-BenchGames7280011666389+32.8
ARC-AGI-2General QA92.561.4−31.1
LAB-Bench ProtocolQA (SecureBio)Science81.951.4−30.5
ChartographyMultimodal45.014.7−30.3
Toloka Arena (v3)Agents61.031.0−30.0
LLM Chess EloGames1550822−29.9
Mystery Game PuzzlesGames58.029.0−29.0
Olam Social Poker EloGames15341508−28.3
SRE Bench — Fully SolvedCode30.53.8−26.7
Terminal-Bench 2.1 Extended (Mercor held-out)Code37.411.8−25.6
RiemannBench (November 2026 rescoring)Math83.257.6−25.6
SM BenchSafety61.186.3+25.2
DeepsecBench — Revalidated V1Code35.410.9−24.5
DeepsecBench — Revalidated V1 (233 issues)Code35.310.9−24.4
Concept Synth INDUCTION Challenge64 Round 1General QA57.834.4−23.4
LLM Creative Story WritingChat & Writing2.50.4−23.0
HPCT (SecureBio)Science76.353.7−22.6
EnterpriseBench: CoreCraftProfessional Work46.268.2+22.0
TaxCalcBench TY25Finance & Accounting30.08.0−22.0
DistressBenchSafety73.652.4−21.2
HardGeoBench Avg DistanceMultimodal14292104−20.8
ALE-BenchCode24101823−20.8
Bio Capabilities Index (SecureBio)Indexes169141−20.2
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.020.0+20.0
Concept Synth INDUCTION Challenge100 Round 1General QA43.024.0−19.0
LAB-Bench CloningScenarios (SecureBio)Science98.880.1−18.7
ArXivMath 06/2026Math88.570.1−18.4
Chess Puzzles (task version 2.0)Games59.041.0−18.0
Extended NYT Connections 940 (Quadratic v1)Games93.876.8−17.0
MetagenomicsBench V0Science31.348.3+17.0
Short Form Bench v1.0Games24102578+16.7
BuseyBench SVGMultimodal6.14.6−15.4
MBCT (SecureBio)Science61.146.1−15.0
CritPt v2 (agentic, avg@5)Science77.162.3−14.9
BrowseComp (Mercor 130-question subset)Search90.675.8−14.8
BrokenArXiv 06/2026Math67.352.8−14.5
ReactBench (39 tasks)Code46.732.3−14.4
ITBench-AAAgents56.242.1−14.1
GeneBench-Pro Public Release SubsetScience24.010.0−14.0
HardGeoBench <=100miMultimodal41.327.5−13.7
FoodTruck BenchGames5322942561−13.7
HardGeoBench <=10miMultimodal30.017.5−12.5
SAGEMath52.640.8−11.8
GBENCH One-shot Coding GScoreGames67.756.3−11.4
Medical Long Context Reasoning (MLCR-AA)Health & Medicine26.115.0−11.1
AA-Briefcase v1.1 EloAgents14361636+11.0
Terminal-Bench-Science 0.1Science22.411.4−11.0
CritPt (avg@5)Science28.618.0−10.6
KernelBench Mega Normalized (RTX PRO 6000)Code7.418.0+10.5
Vision Arena OverallMultimodal12881249−10.0
Terminal-Bench Science 0.1 (Vals)Science20.010.0−10.0
Harvey HLAB — Criteria Pass RateLegal80.590.5+9.9
LMCAGeneral QA58.549.4−9.1
DAYJOB: FinanceFinance & Accounting5.514.5+9.0
WellnessBenchSafety57.048.0−9.0
Roboflow Vision Evals (October 2026, five tasks)Multimodal72.463.5−8.9
ChessBench EloGames12181076−8.4
Toloka Arena (v1)Agents47.639.3−8.3
VIALSMultimodal26.518.2−8.3
MysteryMechanismScience33.325.2−8.1
Code MigrationCode52.944.8−8.1
GDP.pdfProfessional Work30.722.8−7.9
Conceptual Reasoning IndexIndexes68.763.1−7.7
GameBench 2Games80.172.3−7.7
APEX-SWECode45.853.4+7.6
LMArena Text Arena EloChat & Writing14851442−7.5
AA-AnalystAgent (pass^5)Professional Work47.555.0+7.5
VCT v2 (SecureBio)Science49.442.0−7.4
LMArena Agent Arena Net ImprovementAgents6.34.0−7.4
ARC-AGI-1 Semi-PrivateGeneral QA97.590.2−7.3
Senior SWE-BenchCode34.727.4−7.3
SWE-TogetherCode57.865.1+7.3
BrowseComp Extended (Mercor held-out)Search32.339.5+7.2
Ruby LLM Benchmarks — Program FixerCode75.268.0−7.2
Roboflow Vision Evals (September 2026 protocol)Multimodal79.071.9−7.1
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting75.668.6−7.1
CyberBench v1.1 OverallCybersecurity76.369.5−6.8
DAYJOB: HealthcareHealth & Medicine1.68.4+6.8
Humanity's Last Exam DiamondGeneral QA30.423.6−6.8
WCB (SecureBio)Science64.457.6−6.8
dfbench Detection RecallCybersecurity65.759.0−6.7
Vals Science IndexIndexes41.534.8−6.7
WeirdML v3Science15.08.5−6.5
SWE Atlas - Codebase QnACode53.547.0−6.5
GDPval-AA v2.1Professional Work15771717+6.1
Vibe Code Bench v1.1Code80.586.2+5.7
Epoch Capabilities Index (ECI)Indexes163156−5.6
MedCodeHealth & Medicine44.049.6+5.6
Vending-Bench 2Games961910537+5.6
AutomationBench-AA 1.0.6Agents60.165.6+5.5
Excel Modeling BenchmarkFinance & Accounting72.367.0−5.3
ARC-AGI-3 (Semi-Private)General QA7.01.8−5.2
APEX-Agents 1.1 Corporate Law (68-task image-free slice)Legal80.675.6−5.0
AA-Omniscience IndexIndexes22.032.0+5.0
APEX-AccountingFinance & Accounting51.546.7−4.8
CursorBench 4.0Code41.746.3+4.6
Terminal-Bench 2.1Code89.585.0−4.5
MMMU ProMultimodal82.778.2−4.5
Humanity's Last Exam (Text-Only)General QA47.343.1−4.2
MedScribeHealth & Medicine85.289.4+4.1
AA-LCR v1.1Search84.080.0−4.0
LMArena Agent Arena Net Improvement (Steering Burden)Agents4.62.6−3.9
LMArena Agent Arena Steering BurdenAgents5.5-2.3−3.9
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting64.968.7+3.8
HardGeoBench Country AccuracyMultimodal47.543.8−3.7
LLM Debate BenchmarkChat & Writing16451620−3.6
FrontierCUA pass@1Agents16.420.0+3.6
Vals Index v2Indexes63.760.2−3.5
LMArena Agent Arena Bash RecoveryAgents0.77.2+3.2
CheatBenchSafety74.978.0−3.1
Artificial Analysis Progress Index v1Indexes331323−3.1
Finance Agent v2 - ComparablesFinance & Accounting45.342.2−3.1
Vals Index v2.1Indexes58.054.9−3.1
Finance Agent v2 - Financial ModelingFinance & Accounting20.717.7−3.0
LMArena Agent Arena SteerabilityAgents6.50.5−3.0
Finance Agent v2 - AdjustmentsFinance & Accounting49.246.3−3.0
Professional Reasoning Benchmark - LegalLegal50.547.6−2.9
GPQA DiamondGeneral QA89.992.7+2.8
ChessBench CoherenceGames89.186.3−2.8
GPQA-Bio (SecureBio)Science85.382.5−2.8
ObviousBenchGeneral QA96.599.3+2.8
FrontierSWE V2Code32.229.5−2.7
SpeciEval (Refusal-Aware Retries, Historical)Safety99.096.4−2.6
SpeciEval (Pooled English Runs)Safety99.096.4−2.6
LegalBenchLegal87.084.4−2.6
ChessBench AccuracyGames76.474.0−2.6
DeepResearch Bench II (Mercor 130-task set)Search51.549.0−2.5
Finance Agent v2 - Market AnalysisFinance & Accounting64.261.8−2.3
LiveBench 2.0Agents76.374.1−2.2
Eval Connections One-Shot with TrapsGames83.085.0+2.0
Creative Writing v3 EloChat & Writing19742012+2.0
OTIS Mock AIME 2024-2025Math100.098.1−1.9
PubMedQA (SecureBio)Health & Medicine81.179.2−1.9
ACCoRDGeneral QA75.974.0−1.9
BioMysteryBench (Vals Terminus 2)Science71.169.3−1.9
LMArena Agent Arena Praise vs ComplaintAgents6.33.1−1.6
ComplexConstraintsChat & Writing50.548.9−1.6
Arena Alignment IndexSafety84.282.7−1.4
LMArena Agent Arena Confirmed SuccessAgents2.35.1+1.4
Longform Creative WritingChat & Writing81.783.0+1.3
FrontierCode 1.1 Extended - ScoreCode60.659.4−1.2
Finance Agent v2 - Disclosure AnalysisFinance & Accounting63.364.5+1.1
HealthBenchHealth & Medicine55.354.2−1.1
Creative Writing v3 Rubric ScoreChat & Writing16.215.9−1.1
Blueprint-Bench 2Multimodal0.30.3−1.1
Tax Agent BenchFinance & Accounting68.069.0+1.0
LMArena Agent Arena Bash ReliabilityAgents11.49.3−1.0
Terminal-Bench 4.0 (Artificial Analysis)Code24.725.8+1.0
ProgramBenchCode1.50.5−1.0
Veracode GenAI Code Security (Fall 2026)Cybersecurity70.069.0−1.0
Legal Research BenchLegal48.147.1−1.0
AutoResearchExam (hidden-test AUARC)Agents0.50.5+1.0
WMDP-Bio (SecureBio)Science89.788.8−0.9
Public Benefits Bench v1.1Professional Work66.565.6−0.9
Financial Coaching BenchFinance & Accounting74.974.1−0.8
RedlineBenchLegal50.349.5−0.8
Finance Agent v2 - All PassFinance & Accounting41.442.1+0.8
ABC-Bench liquid handling (SecureBio)Science98.798.0−0.7
ForecastBench Preliminary Dataset Brier IndexForecasting61.460.7−0.7
Artificial Analysis Intelligence Index v4.3.2Indexes47.046.4−0.5
CVDP (517-problem set)Code55.454.9−0.5
ATLAS-Finance v1.0Finance & Accounting4.75.0+0.3
Terminal-Bench 4.0Code37.337.6+0.3
DTBenchGeneral QA95.796.0+0.3
DeepSWE v1.1Code70.771.0+0.3
Finance Agent v2 - PrecedentsFinance & Accounting26.225.9−0.2
Finance Agent v2 - Earnings AnalysisFinance & Accounting74.474.6+0.2
MMLU-Bio (SecureBio)Science91.190.9−0.2
Realm Financial ReasoningFinance & Accounting44.444.5+0.1
LMArena Agent Arena Tool HallucinationAgents1.01.00.0
CadQueryEval (Corrected Geometry)Code88.088.00.0
FrontierMath ErdősMath0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.00.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal20.020.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine18.718.70.0
SciCode 1.0.1 (Artificial Analysis)Code57.857.80.0

GPT-5.6 SolvsGrok 4.7normalised score marginsData Nov 15, 2026Data: Vals AI, Artificial Analysis, SecureBio and 57 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…