Atlas

Compare Models

GPT-6.1 Sol vs Grok 4.6

GPT-6.1 Sol leads by 6.3 capability points (≈95% interval +4.0 to +8.6): a clear lead over Grok 4.6 · P(higher) ≈ 100%

Wins 141 of 211 shared benchmarks (49 lost, 21 tied)

Capability vs. relative cost
GPT-6.1 SolGrok 4.6Closed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

GPT-6.1 SolvsGrok 4.6capability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins141 – 49 · 211 shared, 21 tied

Comparing matching setups only (59 benchmarks excluded) · suites counted once
← GPT-6.1 SolGrok 4.6 →

GPT-6.1 SolvsGrok 4.6wins by categoryData Nov 15, 2026Data: Vals AI, Artificial Analysis, Mercor and 48 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Grok 4.6 minus GPT-6.1 Sol.

FrontierMath Tier 4 (v2) - PrivateMath100.031.7−68.3
AA-PersonalityBench Organization indexChat & Writing81.014.3−67.3
Concept Synth INDUCTION Challenge100 Round 1General QA90.025.0−65.0
Design Creativity BenchCode63.743.4−64.6
Concept Synth INDUCTION Challenge64 Round 1General QA95.335.9−59.4
Clipping Bench v1.0Games5707805−52.7
IOI (Vals v2)Code96.947.6−49.3
ProofBench v1.1Math99.051.0−48.0
LLM Chess EloGames1612443−48.0
Olam Social Poker EloGames15431499−47.8
Terminal-Bench-Science 0.1Science50.52.9−47.6
BrowseComp Extended (Mercor held-out)Search67.721.5−46.2
Mystery Game PuzzlesGames80.034.0−46.0
Gray Swan IPI Attack Success (k=15)Safety8.651.8−43.2
AA-PersonalityBench Imagination indexChat & Writing61.920.0−42.3
Terminal-Bench Science 0.1 (Vals)Science52.911.4−41.4
ALE-BenchCode29901822−41.4
Furniture Assembly BenchmarkMultimodal80.040.0−40.0
ChessBench EloGames1562891−39.9
AA-PersonalityBench Conviction indexChat & Writing54.392.4+38.5
ARC-AGI-3 (Semi-Private)General QA39.92.1−37.8
Humanity's Last Exam DiamondGeneral QA58.821.5−37.3
CheatBenchSafety42.879.6−36.8
VoxelBench Text RatingGames25502050−35.6
MMMU-Pro Extended (Mercor held-out)Multimodal61.326.0−35.3
Chess Puzzles (task version 2.0)Games68.033.0−35.0
AA-PersonalityBench Warmth indexChat & Writing65.731.5−34.6
NoCoT-Bench NCRI 15.2General QA14986.8−34.2
BuseyBench SVGMultimodal8.04.6−33.4
SWE-bench Verified Extended (Mercor held-out)Code45.378.0+32.7
Terminal-Bench 4.0 (Artificial Analysis)Code51.521.2−30.3
VIALSMultimodal49.120.5−28.6
CyberBench v1.1 OverallCybersecurity39.367.7+28.4
Terminal-Bench 4.0Code53.425.3−28.1
FrontierMath Tiers 1-3 (v2) - PrivateMath93.766.0−27.7
ARC-AGI-2General QA94.267.1−27.1
ChessBenchGames68.743.3−25.3
Wiz Cyber Model Arena v3.9Cybersecurity70.745.5−25.2
SimpleQA Verified (Epoch anti-abstention)General QA73.949.3−24.6
CrossSearch (pass^3)Search59.035.0−24.0
LMArena Agent Arena Net Improvement (Steering Burden)Agents12.5-0.2−23.9
GBENCH One-shot Coding GScoreGames79.055.4−23.7
Artificial Analysis Intelligence Index v5.0.1Indexes54.631.0−23.6
DeepsecBench — Revalidated V1 (233 issues)Code39.516.1−23.4
GameBench 2Games92.068.7−23.3
WellnessBenchSafety66.043.0−23.0
RareDiseaseBench V1 (35 evaluations)Science27.64.8−22.9
Roboflow Vision Evals (October 2026, five tasks)Multimodal81.759.1−22.6
YC-BenchGames9362671568241+22.1
Medical Long Context Reasoning (MLCR-AA)Health & Medicine33.912.2−21.7
Vals Science IndexIndexes59.638.1−21.5
AA-PersonalityBench Slop indexChat & Writing14.335.3+21.2
AA-PersonalityBench Transparency indexChat & Writing92.471.4−21.2
Code MigrationCode65.144.6−20.5
CritPt v2 (agentic, avg@5)Science82.662.3−20.3
SM BenchSafety69.089.0+20.0
SnorkelLegalLegal20.740.7+20.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal20.040.0+20.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.020.0+20.0
HealthBench ProfessionalHealth & Medicine68.348.5−19.8
TissueMicroscopyBench V1 (49 evaluations)Science22.42.7−19.7
FrontierCode 2.0 (Artificial Analysis)Code38.519.8−18.8
SnorkelFinance 2.0Finance & Accounting6.625.1+18.5
ChessBench CoherenceGames87.569.2−18.3
GeneBench-Pro Public Release SubsetScience33.015.0−18.0
GDP.pdfProfessional Work35.017.2−17.8
SAGEMath46.528.9−17.6
HardGeoBench Avg DistanceMultimodal12391807−17.5
AA-PersonalityBench Informativeness indexChat & Writing27.744.8+17.3
Roboflow Vision Evals (September 2026 protocol)Multimodal85.568.7−16.8
MedXpertQA MM Extended (Mercor held-out)Health & Medicine64.748.4−16.3
MysteryMechanismScience46.430.6−15.8
Extended NYT Connections 940 (Quadratic v1)Games95.580.0−15.5
EnterpriseBench: CoreCraftProfessional Work50.365.6+15.3
HardGeoBench <=10miMultimodal35.020.0−15.0
LMArena Agent Arena Praise vs ComplaintAgents27.5-2.5−15.0
LMCAGeneral QA62.848.5−14.3
KernelBench Mega Normalized (RTX PRO 6000)Code14.10.0−14.1
Conceptual Reasoning IndexIndexes72.762.5−14.0
ChessBench AccuracyGames80.767.8−13.8
SpatialBench-Long V1 Clean RerunsScience39.926.2−13.7
AA-PersonalityBench Expressiveness indexChat & Writing50.563.8+13.5
Vibe Code Bench v1.1Code88.976.2−12.7
NoCoT-Bench NCKI (kspine v3)General QA130112−12.4
SUPERChemScience77.865.7−12.1
CritPt (avg@5)Science31.719.7−12.0
SWE Atlas Codebase QnA Extended (Mercor held-out)Code31.319.3−12.0
BrowseComp (Mercor 130-question subset)Search95.484.0−11.4
Design Arena Overall Code CategoriesCode13561288−11.3
HardGeoBench <=100miMultimodal43.832.5−11.2
HieroglyphBenchMultimodal31.420.3−11.1
ARC-AGI-1 Semi-PrivateGeneral QA98.587.5−11.0
APEX-SWECode46.957.7+10.8
Bedside BenchHealth & Medicine93.983.3−10.6
MedXpertQA MM (200-question set)Health & Medicine85.775.2−10.5
LiveBench 2.0Agents85.474.9−10.5
ScBench-Long V1 Clean RerunsScience26.716.5−10.2
Toloka Arena (v3)Agents66.056.0−10.0
Humanity's Last Exam (Text-Only)General QA52.642.9−9.7
Legal Research BenchLegal38.548.1+9.6
Artificial Analysis Progress Index v1Indexes348322−9.5
BioSecBench-Refusal V1 Clean RerunsSafety37.646.9+9.3
ACCoRDGeneral QA80.971.6−9.3
Snorkel Agentic Coding 2.0Code42.833.6−9.2
Tax Agent BenchFinance & Accounting62.371.4+9.1
Vals Index v2.1Indexes61.252.1−9.1
Epoch Capabilities Index (ECI)Indexes169158−8.9
AA-AnalystAgent (pass^5)Professional Work50.041.3−8.7
HardGeoBench Country AccuracyMultimodal52.543.8−8.7
DeepSWE v1.1Code75.266.7−8.5
Excel Modeling BenchmarkFinance & Accounting70.862.7−8.0
CharXiv Extended (Mercor held-out)Multimodal85.177.3−7.8
SWE-bench MultilingualCode84.992.7+7.8
LMArena Agent Arena Steering BurdenAgents13.4-1.8−7.6
Artificial Analysis Intelligence Index v4.3.2Indexes51.844.3−7.5
Vision Arena OverallMultimodal12931263−7.5
Public Benefits Bench v1.1Professional Work59.366.8+7.5
BioMysteryBench (Vals Terminus 2)Science79.672.2−7.4
SnorkelUnderwrite 2.0Finance & Accounting25.017.8−7.2
Arena Alignment IndexSafety87.980.8−7.2
LMArena Agent Arena Confirmed SuccessAgents11.7-2.6−7.2
MetagenomicsBench V1Science45.738.7−7.0
SimpleBenchGeneral QA82.975.9−7.0
SpeciEval (Refusal-Aware Retries, Historical)Safety97.490.8−6.5
SpeciEval (Pooled English Runs)Safety97.490.8−6.5
TxBench Oligo Discovery V1 (113 evaluations)Science52.546.0−6.5
DAXBench v2Code92.686.2−6.4
τ³-Banking (Fireworks, banking_knowledge)Agents44.750.7+6.0
BioSecBench-Structure V1 (58 evaluations)Science21.315.5−5.7
VariantBench V1 Clean RerunsScience46.040.4−5.6
KernelBench CUDA Normalized (RTX PRO 6000)Code43.237.7−5.5
AA-Omniscience IndexIndexes41.530.5−5.5
Access to Justice BenchmarkLegal65.871.3+5.5
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine24.018.7−5.3
Harvey LAB: In-House Contracting (held-out)Legal4.710.0+5.3
DAYJOB: FinanceFinance & Accounting18.012.8−5.2
DAYJOB: HealthcareHealth & Medicine7.62.8−4.8
Harvey HLAB — Criteria Pass RateLegal87.892.5+4.7
EpiBench V1 Clean RerunsScience28.323.6−4.7
MMMU ProMultimodal85.180.5−4.6
LMArena Text Arena EloChat & Writing14821456−4.5
ScBench V1 Clean RerunsScience63.659.1−4.4
GDPval-AA v2.1Professional Work15631658+4.2
MedCodeHealth & Medicine48.844.7−4.1
Snorkel SWE-bench CLICode9.85.7−4.1
APEX-Agents 1.1Professional Work73.277.3+4.1
APEX-Accounting 1.1Finance & Accounting64.360.3−4.0
SnorkelRevOpsProfessional Work12.115.8+3.7
TxBench Antibody Discovery V1 (100 evaluations)Science49.753.0+3.3
BioSecBench-Surveillance V1 Clean RerunsScience50.847.5−3.3
SnorkelManufacturingProfessional Work19.616.4−3.2
CharXiv (descriptive and reasoning combined)Multimodal96.092.9−3.1
AA-LCR v1.1Search84.081.0−3.0
Terminal-Bench 2.1Code85.488.4+3.0
CVDP (517-problem set)Code55.158.0+2.9
SWE-bench VerifiedCode87.384.6−2.7
τ³-Banking (Mercor run, bm25_grep retrieval)Agents43.045.7+2.7
LMArena Agent Arena Bash ReliabilityAgents5.010.0+2.5
SWE-TogetherCode58.360.6+2.3
Snorkel WorkplaceAgentsProfessional Work15.017.1+2.1
ProgramBenchCode3.01.0−2.0
TxBench Preclinical Pharmacology V1 Clean RerunsScience59.761.7+2.0
BioSecBench-Function V1 Clean RerunsScience46.044.0−2.0
AA-Briefcase v1.1 EloAgents15041540+1.9
SWE Atlas - Codebase QnACode48.450.3+1.9
Immigration BenchLegal67.866.0−1.8
TxBench-Antibody-Development V1 (60 evaluations)Science27.826.1−1.7
Finance Agent v2Finance & Accounting52.053.7+1.6
MMLU ProGeneral QA91.089.4−1.6
AdvancedIFChat & Writing82.480.9−1.5
NeuroscienceBench V1 (91 evaluations)Science22.020.5−1.5
MacroscopeBenchCode77.275.9−1.3
BioSecBench-RespiratoryVirus-Surveillance V1 (110 evaluations)Science12.411.5−0.9
OTIS Mock AIME 2024-2025Math100.099.2−0.8
DTBenchGeneral QA96.597.3+0.8
ForecastBench Preliminary Dataset Brier IndexForecasting61.761.0−0.7
SciCode 1.0.1 (Artificial Analysis)Code55.856.5+0.7
Snorkel Digital Work IndexIndexes21.220.5−0.7
LMArena Agent Arena Bash RecoveryAgents4.55.7+0.6
Prophet Arena Arena Score (Agentic, OpenCode)Forecasting86.085.6−0.4
AutomationBench-AA 1.0.6Agents66.667.0+0.4
Terminal-Bench 2.1 Extended (Mercor held-out)Code45.845.5−0.3
Prophet Arena Arena Score (Fixed Context)Forecasting84.885.0+0.1
MedScribeHealth & Medicine86.586.5+0.1
LMArena Agent Arena Tool HallucinationAgents1.01.00.0
AIME 2025Math100.0100.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal20.020.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal20.020.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
ObviousBenchGeneral QA100.0100.00.0
SciCode (main problems)Science14.614.60.0
SpatialBench V1 Clean RerunsScience67.367.30.0

GPT-6.1 SolvsGrok 4.6normalised score marginsData Nov 15, 2026Data: Vals AI, Artificial Analysis, Mercor and 48 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…