Atlas

Compare Models

GPT-6 Sol vs Grok 4.6

GPT-6 Sol leads by 3.8 capability points (≈95% interval +1.6 to +5.9): a clear lead over Grok 4.6 · P(higher) ≈ 100%

Wins 133 of 217 shared benchmarks (66 lost, 18 tied)

Capability vs. relative cost
GPT-6 SolGrok 4.6Closed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

GPT-6 SolvsGrok 4.6capability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins133 – 66 · 217 shared, 18 tied

Comparing matching setups only (79 benchmarks excluded) · suites counted once
← GPT-6 SolGrok 4.6 →

GPT-6 SolvsGrok 4.6wins by categoryData Nov 15, 2026Data: Vals AI, Mercor, Artificial Analysis and 49 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Grok 4.6 minus GPT-6 Sol.

FrontierMath Tier 4 (v2) - PrivateMath90.031.7−58.3
Olam Social Poker EloGames15421499−46.7
LLM Chess EloGames1545443−45.3
AA-PersonalityBench Conviction indexChat & Writing50.592.4+42.3
Gray Swan IPI Attack Success (k=15)Safety10.151.8−41.7
BrowseComp Extended (Mercor held-out)Search63.221.5−41.7
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal0.040.0+40.0
SWE-bench Verified Extended (Mercor held-out)Code39.378.0+38.7
IOI (Vals v2)Code82.647.6−35.0
AA-PersonalityBench Informativeness indexChat & Writing10.544.8+34.6
AA-PersonalityBench Slop indexChat & Writing1.035.3+34.6
ChartographyMultimodal53.620.5−33.1
AA-PersonalityBench Transparency indexChat & Writing39.171.4+32.7
Vending-Bench 2Games144289047−32.6
ProofBench v1.1Math83.051.0−32.0
AA-PersonalityBench Expressiveness indexChat & Writing33.463.8+30.8
BuseyBench SVGMultimodal7.54.6−29.2
ALE-BenchCode26371822−28.9
CadQueryEval (Corrected Geometry)Code100.072.0−28.0
MMMU-Pro Extended (Mercor held-out)Multimodal51.926.0−25.9
LMArena Agent Arena Net ImprovementAgents8.81.1−25.3
DeepsecBench — Revalidated V1Code40.916.2−24.7
DeepsecBench — Revalidated V1 (233 issues)Code40.716.1−24.7
ChessBench EloGames1303891−24.5
VoxelBench Text RatingGames23942050−24.5
ZeroBench (Main questions, pass@5)Multimodal41.017.0−24.0
FrontierMath Tiers 1-3 (v2) - PrivateMath89.866.0−23.9
Wiz Cyber Model Arena v3.9Cybersecurity69.345.5−23.8
Humanity's Last Exam DiamondGeneral QA45.221.5−23.7
AA-PersonalityBench Warmth indexChat & Writing54.331.5−23.1
SM BenchSafety66.089.0+23.0
Chess Puzzles (task version 2.0)Games56.033.0−23.0
LAB-Bench LitQA2 (SecureBio)Science51.828.9−22.9
ARC-AGI-2General QA89.667.1−22.5
EnterpriseBench: CoreCraftProfessional Work43.165.6+22.5
Mystery Game PuzzlesGames56.034.0−22.0
Harvey LAB held-out set (Fireworks run)Legal0.722.1+21.4
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.020.0+20.0
MedXpertQA MM Extended (Mercor held-out)Health & Medicine68.048.4−19.6
Legal Research BenchLegal28.848.1+19.2
ChessBench CoherenceGames88.269.2−19.0
ChessBenchGames62.043.3−18.7
CritPt v2 (agentic, avg@5)Science80.962.3−18.6
Terminal-Bench Science 0.1 (Vals)Science30.011.4−18.6
Terminal-Bench 4.0Code43.825.3−18.5
Tax Agent BenchFinance & Accounting53.071.4+18.3
Furniture Assembly BenchmarkMultimodal58.340.0−18.3
Bio Capabilities Index (SecureBio)Indexes161137−17.8
Roboflow Vision Evals (October 2026, five tasks)Multimodal76.959.1−17.8
ZeroBench Main Mean pass@1 (five samples)Multimodal28.811.0−17.8
SAGEMath44.828.9−15.9
FrontierCode 2.0 (Artificial Analysis)Code35.619.8−15.8
NoCoT-Bench NCRI 15.2General QA11586.8−15.3
Veracode GenAI Code Security (Fall 2026)Cybersecurity73.058.0−15.0
GBENCH One-shot Coding GScoreGames70.155.4−14.7
ComplexConstraintsChat & Writing50.836.5−14.3
Roboflow Vision Evals (September 2026 protocol)Multimodal82.368.7−13.6
LMArena Agent Arena Net Improvement (Steering Burden)Agents6.9-0.2−13.4
Artificial Analysis Intelligence Index v5.0.1Indexes44.231.0−13.1
WellnessBenchSafety56.043.0−13.0
LAB-Bench ProtocolQA (SecureBio)Science81.168.1−13.0
HealthBench ProfessionalHealth & Medicine61.248.5−12.7
APEX-SWECode45.057.7+12.7
Code MigrationCode57.244.6−12.6
SWE Atlas Codebase QnA Extended (Mercor held-out)Code31.319.3−12.0
ScBench-Long V1 Clean RerunsScience28.416.5−11.9
Conceptual Reasoning IndexIndexes71.062.5−11.7
Vibe Code Bench v1.1Code87.876.2−11.6
SimpleQA Verified (Epoch anti-abstention)General QA60.749.3−11.4
CritPt (avg@5)Science30.919.7−11.1
GeneBench-Pro Public Release SubsetScience26.015.0−11.0
SWE-bench MultilingualCode81.792.7+11.0
Terminal-Bench-Science 0.1Science17.16.2−11.0
ACCoRDGeneral QA82.471.6−10.8
GameBench 2Games79.468.7−10.7
MBCT (SecureBio)Science61.150.4−10.7
LMCAGeneral QA59.148.5−10.6
CyberBench v1.1 OverallCybersecurity78.067.7−10.3
ChessBench AccuracyGames77.467.8−10.3
Public Benefits Bench v1.1Professional Work56.666.8+10.2
WCB (SecureBio)Science60.550.3−10.2
Extended NYT Connections 940 (Quadratic v1)Games90.180.0−10.1
ZeroBench Main Questions pass^5Multimodal17.07.0−10.0
AA-Briefcase v1.1 EloAgents13591540+9.9
AA-PersonalityBench Imagination indexChat & Writing29.620.0−9.6
HardGeoBench Avg DistanceMultimodal14971807−9.6
Harvey HLAB — Criteria Pass RateLegal83.592.5+9.0
BioSecBench-Refusal V1 Clean RerunsSafety37.946.9+9.0
Excel Modeling BenchmarkFinance & Accounting71.562.7−8.8
Harvey LAB: In-House Contracting (held-out)Legal1.310.0+8.7
APEX-Agents 1.1Professional Work68.777.3+8.6
GDPval-AA v2.1Professional Work14661658+8.5
ARC-AGI-1 Semi-PrivateGeneral QA95.587.5−8.0
SpeciEval (Refusal-Aware Retries, Historical)Safety98.590.8−7.6
SpeciEval (Pooled English Runs)Safety98.590.8−7.6
TxBench Antibody Discovery V1 Clean RerunsScience37.044.5+7.6
GDP.pdf (Artificial Analysis, All-pass)Professional Work25.217.8−7.4
SUPERChemScience72.865.7−7.1
LiveBench 2.0Agents81.974.9−7.0
Vals Science IndexIndexes45.038.1−6.9
Arena Alignment IndexSafety87.680.8−6.9
CharXiv Extended (Mercor held-out)Multimodal83.877.3−6.5
HardGeoBench <=10miMultimodal26.320.0−6.2
LMArena Agent Arena Praise vs ComplaintAgents10.0-2.5−6.2
HPCT (SecureBio)Science67.861.6−6.2
SWE Atlas - Codebase QnACode44.150.3+6.2
MedXpertQA MM (200-question set)Health & Medicine81.375.2−6.1
LMArena Agent Arena Steering BurdenAgents10.1-1.8−6.0
VCT (SecureBio)Science54.048.3−5.7
HieroglyphBenchMultimodal26.020.3−5.7
LMArena Agent Arena SteerabilityAgents14.53.3−5.6
Vals Index v2.1Indexes57.552.1−5.4
BrowseComp (Mercor 130-question subset)Search89.484.0−5.4
SpatialBench-Long V1 Clean RerunsScience31.626.2−5.4
AutomationBench-AA 1.0.6Agents61.767.0+5.3
CVDP (517-problem set)Code52.758.0+5.3
Terminal-Bench 4.0 (Artificial Analysis)Code26.321.2−5.1
AA-LCR Extended (Mercor held-out)Search70.265.2−5.0
HardGeoBench <=100miMultimodal37.532.5−5.0
Epoch Capabilities Index (ECI)Indexes164158−4.9
NoCoT-Bench NCKI (kspine v3)General QA119112−4.9
Bedside BenchHealth & Medicine88.183.3−4.8
BioSecBench-Function V1 Clean RerunsScience39.344.0+4.8
Clipping Bench v1.0Games1240805−4.7
Finance Agent v2Finance & Accounting49.053.7+4.6
MedScribeHealth & Medicine82.086.5+4.5
YC-BenchGames14417931568241+4.4
Artificial Analysis Progress Index v1Indexes334322−4.4
KernelBench CUDA Normalized (RTX PRO 6000)Code42.037.7−4.3
LAB-Bench CloningScenarios (SecureBio)Science98.594.2−4.3
SWE-bench VerifiedCode80.384.6+4.3
APEX-1 (Mercor, mean score)Professional Work63.667.7+4.1
Medical Long Context Reasoning (MLCR-AA)Health & Medicine16.112.2−3.9
Blueprint-Bench 2Multimodal0.40.3−3.7
τ³-Banking (Fireworks, banking_knowledge)Agents47.050.7+3.7
LMArena Agent Arena Confirmed SuccessAgents4.5-2.6−3.6
DAYJOB: FinanceFinance & Accounting9.312.8+3.5
Vals Index v2Indexes62.659.2−3.4
Humanity's Last Exam (Text-Only)General QA46.342.9−3.4
MetagenomicsBench V1Science42.038.7−3.3
Artificial Analysis Intelligence Index v4.3.2Indexes47.644.3−3.3
MetagenomicsBench V0Science42.038.7−3.3
GlobeBenchGeneral QA25.028.1+3.1
Terminal-Bench 2.1 Extended (Mercor held-out)Code42.445.5+3.1
GDPval Extended (Mercor held-out rubric score)Professional Work58.761.8+3.1
Terminal-Bench 2.1Code85.488.4+3.0
SimpleBenchGeneral QA73.175.9+2.8
TxBench Oligo Discovery V1 Clean RerunsScience40.643.3+2.8
τ³-Banking (Mercor run, bm25_grep retrieval)Agents43.045.7+2.7
KernelBench Mega Normalized (RTX PRO 6000)Code2.70.0−2.7
AA-LCR v1.1Search83.781.0−2.7
BioMysteryBench (Vals Terminus 2)Science74.872.2−2.6
GDPval (Mercor 213-task rubric score)Professional Work83.586.0+2.5
HardGeoBench Country AccuracyMultimodal46.343.8−2.5
ScBench V1 Clean RerunsScience61.559.1−2.4
MedCodeHealth & Medicine47.144.7−2.4
CharXiv (descriptive and reasoning combined)Multimodal95.092.9−2.1
MMMU ProMultimodal82.580.5−2.0
Eval Connections One-Shot with TrapsGames89.091.0+2.0
Toloka Arena (v3)Agents58.056.0−2.0
AA-PersonalityBench Organization indexChat & Writing12.414.3+1.9
SciCode (main problems)Science12.714.6+1.9
BioSecBench-Structure V1 (58 evaluations)Science13.815.5+1.7
VariantBench V1 Clean RerunsScience38.740.4+1.7
AA-Omniscience IndexIndexes27.130.5+1.7
ForecastBench Preliminary Dataset Brier IndexForecasting62.661.0−1.6
EpiBench V1 Clean RerunsScience22.023.6+1.6
GPQA-Bio (SecureBio)Science88.987.4−1.5
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine20.018.7−1.3
TxBench Preclinical Pharmacology V1 Clean RerunsScience63.061.7−1.3
MacroscopeBenchCode77.275.9−1.3
LibraryDesignBenchCode38.539.7+1.2
SciCode 1.0.1 (Artificial Analysis)Code57.656.5−1.2
MMLU-Bio (SecureBio)Science90.891.8+1.0
ProgramBenchCode2.01.0−1.0
SAB ResearchFinance & Accounting95.694.6−1.0
OTIS Mock AIME 2024-2025Math100.099.2−0.8
Design Arena Overall Code CategoriesCode12931288−0.8
DAYJOB: HealthcareHealth & Medicine3.62.8−0.8
ObviousBenchGeneral QA99.3100.0+0.7
LMArena Agent Arena Bash RecoveryAgents7.05.7−0.7
APEX-Accounting 1.1Finance & Accounting59.760.3+0.6
MMLU ProGeneral QA88.989.4+0.5
LMArena Agent Arena Bash ReliabilityAgents10.910.0−0.5
MysteryMechanismScience30.230.6+0.5
ARC-AGI-3 (Semi-Private)General QA1.82.1+0.3
SpatialBench V1 Clean RerunsScience67.567.3−0.3
BioSecBench-Surveillance V1 Clean RerunsScience47.747.5−0.2
DeepSWE v1.1Code66.666.7+0.1
LMArena Text Arena EloChat & Writing14571456−0.1
LMArena Agent Arena Tool HallucinationAgents1.01.00.0
AdvancedIFChat & Writing80.980.90.0
AIME 2025Math100.0100.00.0
DTBenchGeneral QA97.397.30.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
SWE-TogetherCode60.660.60.0

GPT-6 SolvsGrok 4.6normalised score marginsData Nov 15, 2026Data: Vals AI, Mercor, Artificial Analysis and 49 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…