Atlas

Compare Models

GPT-6.1 Sol vs Grok 4.7

GPT-6.1 Sol leads by 5.7 capability points (≈95% interval +3.3 to +8.0): a clear lead over Grok 4.7 · P(higher) ≈ 100%

Wins 189 of 258 shared benchmarks (49 lost, 20 tied)

Capability vs. relative cost
GPT-6.1 SolGrok 4.7Closed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

GPT-6.1 SolvsGrok 4.7capability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins189 – 49 · 258 shared, 20 tied

Comparing matching setups only (109 benchmarks excluded) · suites counted once
← GPT-6.1 SolGrok 4.7 →

GPT-6.1 SolvsGrok 4.7wins by categoryData Nov 15, 2026Data: Vals AI, Artificial Analysis, Mercor and 59 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Grok 4.7 minus GPT-6.1 Sol.

ARC-AGI-3 (Semi-Private)General QA95.010.0−84.9
FrontierMath Tier 4 (v2) - PrivateMath100.017.1−82.9
ProofBench v1.1Math99.026.0−73.0
Concept Synth INDUCTION Challenge100 Round 1General QA90.024.0−66.0
ViBench 2.0Code64.71.5−63.2
Concept Synth INDUCTION Challenge64 Round 1General QA95.334.4−60.9
Furniture Assembly BenchmarkMultimodal80.020.8−59.2
Clipping Bench v1.0Games5707384−57.3
AA-PersonalityBench Organization indexChat & Writing81.029.6−51.9
Mystery Game PuzzlesGames80.029.0−51.0
WARC-Bench (192-task test split)Agents86.335.6−50.7
BrokenArXiv 05/2026Math90.040.0−50.0
BrokenArXiv 06/2026Math100.052.8−47.2
SRE Bench — Fully SolvedCode50.83.8−46.9
SheetBench-50Finance & Accounting60.514.3−46.2
Terminal-Bench Science 0.1 (Vals)Science52.910.0−42.9
DeepSWE v1.1Code75.233.3−41.9
Agent Initiative BenchmarkAgents8.74.5−41.4
ALE-BenchCode29901823−41.3
FrontierMath Tiers 1-3 (v2) - PrivateMath93.753.0−40.7
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.040.0+40.0
IOI (Vals v2)Code96.957.7−39.2
Terminal-Bench-Science 0.1Science50.511.4−39.0
Olam Social Poker EloGames15431508−38.0
Short Form Bench v1.0Games29492578−36.8
Humanity's Last Exam DiamondGeneral QA58.823.6−35.2
CheatBenchSafety42.878.0−35.2
Toloka Arena (v3)Agents66.031.0−35.0
AA-PersonalityBench Transparency indexChat & Writing92.458.1−34.6
ArXivMath 05/2026Math95.060.8−34.2
OpenWorldEngineer (Mean@8)Code41.27.0−34.1
Terminal-Bench 2.1 Extended (Mercor held-out)Code45.811.8−34.0
BuseyBench SVGMultimodal8.04.6−33.8
ARC-AGI-2General QA94.261.4−32.8
LLM Chess EloGames1612822−32.4
VIALSMultimodal49.118.2−30.9
SAFE-Teen 106 worst branch: Health Advice (no prompt)Safety8.839.2−30.4
CyberBench v1.1 OverallCybersecurity39.369.5+30.2
ATLAS (Exa, 200-task subset)Search69.539.5−30.0
TaxCalcBench TY25Finance & Accounting38.08.0−30.0
WeirdML v3Science38.38.5−29.8
SAFE-Teen 106 worst branch: Urgent Safety (no prompt)Safety15.444.4−29.1
ChessBench EloGames15621076−28.9
AA-PersonalityBench Conviction indexChat & Writing54.382.9+28.8
SAFE-Teen 106 worst branch: Relationships (no prompt)Safety19.147.9−28.8
DeepsecBench — Revalidated V1 (233 issues)Code39.510.9−28.6
Terminal-Bench 4.0 (Artificial Analysis)Code54.025.8−28.3
BrowseComp Extended (Mercor held-out)Search67.739.5−28.2
Chess Puzzles (task version 2.0)Games68.041.0−27.0
AA-PersonalityBench Imagination indexChat & Writing61.935.3−26.9
HardGeoBench Avg DistanceMultimodal12392104−26.7
FrontierCUA pass@1Agents46.720.0−26.7
HoneyBench v0.2 (hack rate)Safety59.585.1−25.6
YC-BenchGames9362671666389+25.5
AA-PersonalityBench Expressiveness indexChat & Writing50.575.3+25.0
DataCenterBench Tier 1Agents38.313.3−25.0
Vals Science IndexIndexes59.634.8−24.8
ArXivMath 06/2026Math94.470.1−24.3
Code Migration v1.1Code77.353.2−24.1
GeneBench-Pro Public Release SubsetScience33.010.0−23.0
GBENCH One-shot Coding GScoreGames79.056.3−22.7
SAFE-Teen Relationships (no prompt)Safety10.032.5−22.5
DistressBenchSafety74.252.4−21.8
sudo L7 (October 2026 rescoring)Code33.912.2−21.7
RiemannBench (November 2026 rescoring)Math79.257.6−21.6
SAFE-Teen 106: Relationships (no prompt)Safety10.231.5−21.3
MysteryMechanismScience46.425.2−21.2
AA-PersonalityBench Warmth indexChat & Writing65.744.8−21.2
Code MigrationCode65.144.8−20.3
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal20.00.0−20.0
Code2Video-AA EloCode13691127−19.7
GameBench 2Games92.072.3−19.7
BrowseComp (Mercor 130-question subset)Search95.475.8−19.6
Artificial Analysis Intelligence Index v5.0Indexes54.535.3−19.2
Artificial Analysis Intelligence Index v5.0.1Indexes54.635.5−19.1
Medical Long Context Reasoning (MLCR-AA)Health & Medicine33.915.0−18.9
LMArena Agent Arena Net Improvement (Steering Burden)Agents12.52.6−18.7
Extended NYT Connections 940 (Quadratic v1)Games95.576.8−18.7
AA-3D v1.0 EloCode1121992−18.7
Roboflow Vision Evals (October 2026, five tasks)Multimodal81.763.5−18.2
WellnessBenchSafety66.048.0−18.0
EnterpriseBench: CoreCraftProfessional Work50.368.2+17.9
SimpleQA Verified (Epoch anti-abstention)General QA73.956.0−17.9
Hermes IndexIndexes57.139.3−17.8
Artificial Analysis Engineering IndexIndexes59.441.8−17.6
HardGeoBench <=10miMultimodal35.017.5−17.5
SM BenchSafety69.086.3+17.4
MedXpertQA MM (200-question set)Health & Medicine85.769.0−16.7
CritPt v2 (agentic, avg@5)Science78.662.3−16.3
HardGeoBench <=100miMultimodal43.827.5−16.2
SWE Atlas Codebase QnA Extended (Mercor held-out)Code31.315.3−16.0
Terminal-Bench 4.0Code53.437.6−15.8
HANDBOOK.md Agents v1.1Professional Work28.644.0+15.4
Design Arena Overall Code CategoriesCode13561270−14.3
SAFE-Teen 106: Health Advice (no prompt)Safety4.718.7−14.0
SAFE-Teen 106 worst branch: Urgent Safety (under-18 prompt)Safety12.726.7−14.0
CritPt (avg@5)Science31.718.0−13.7
SnorkelFinance 2.0Finance & Accounting6.620.2+13.6
Roboflow Vision Evals (September 2026 protocol)Multimodal85.571.9−13.6
LMCAGeneral QA62.849.4−13.4
FrontierMemory v1.1Agents51.137.8−13.3
Conceptual Reasoning IndexIndexes72.763.1−13.2
MedXpertQA MM Extended (Mercor held-out)Health & Medicine64.752.0−12.7
FrontierCode 1.1 Main (Artificial Analysis)Code46.634.2−12.5
LMArena Agent Arena Praise vs ComplaintAgents27.53.1−12.2
SAFE-Teen Urgent Safety (no prompt)Safety3.515.7−12.2
GDP.pdfProfessional Work35.022.8−12.2
SnorkelLegalLegal20.732.8+12.1
HLAB-Vals-1.1Legal6.318.3+12.1
DAYJOB: Data ScienceProfessional Work18.76.7−12.0
SAFE-Teen Health Advice (no prompt)Safety4.216.1−11.9
SAFE-Teen 106: Urgent Safety (no prompt)Safety3.415.1−11.7
SAFE-Teen 106 worst branch: Health Advice (under-18 prompt)Safety7.719.2−11.5
LiveBench 2.0Agents85.474.1−11.3
Vision Arena OverallMultimodal12931249−11.2
Snorkel Agentic Coding 2.0Code42.831.9−10.9
Harvey LAB (Mercor run, undisclosed ~213-task cohort)Legal8.819.5+10.7
Artificial Analysis Economics IndexIndexes56.345.7−10.6
τ³-Banking (Mercor run, bm25_grep retrieval)Agents43.032.6−10.4
BioMysteryBench (Vals Terminus 2)Science79.669.3−10.4
Epoch Capabilities Index (ECI)Indexes169156−10.4
SWE-bench Verified Extended (Mercor held-out)Code45.355.3+10.0
FrontierCode 2.0 (Artificial Analysis)Code38.528.7−9.9
Humanity's Last Exam (Text-Only)General QA52.643.1−9.5
o11y-bench 1.1 Pass^3Agents85.976.6−9.4
ComplexConstraints (November 2026 rescoring)Chat & Writing70.160.8−9.3
Artificial Analysis Progress Index v1Indexes348323−9.1
APEX-Accounting 1.1Finance & Accounting64.355.3−9.0
HardGeoBench Country AccuracyMultimodal52.543.8−8.7
SAFE-Teen 106: Health Advice (under-18 prompt)Safety3.312.0−8.7
Legal Research BenchLegal38.547.1+8.7
SAFE-Teen Health Advice (under-18 prompt)Safety2.010.6−8.6
ARC-AGI-1 Semi-PrivateGeneral QA98.590.2−8.3
SWE-bench VerifiedCode87.379.1−8.2
SAFE-Teen 106: Urgent Safety (under-18 prompt)Safety2.010.1−8.1
SAFE-Teen Urgent Safety (under-18 prompt)Safety2.110.1−8.1
Access to Justice BenchmarkLegal65.873.7+7.9
LMArena Agent Arena Steering BurdenAgents13.4-2.3−7.8
Westworld Finance Diligence BenchFinance & Accounting55.047.2−7.8
MKA (Artificial Analysis)Professional Work47.740.1−7.6
SnorkelManufacturingProfessional Work19.612.1−7.5
AA-Briefcase v1.1 EloAgents15041636+7.2
ChessBench AccuracyGames80.774.0−7.2
Ruby LLM Benchmarks — Program FixerCode75.068.0−7.0
LMArena Text Arena EloChat & Writing14821442−7.0
MMMU ProMultimodal85.178.2−6.9
CharXiv Extended (Mercor held-out)Multimodal85.178.2−6.9
DualEntry Accounting AI BenchmarkFinance & Accounting81.288.1+6.9
ACCoRDGeneral QA80.974.0−6.8
SWE-TogetherCode58.365.1+6.8
GDPval-AA v2.1Professional Work15631717+6.8
Tax Agent BenchFinance & Accounting62.369.0+6.7
APEX-SWECode46.953.4+6.5
Artificial Analysis Legal IndexIndexes53.747.2−6.5
Public Benefits Bench v1.1Professional Work59.365.6+6.3
Vals Index v2.1Indexes61.254.9−6.2
AA-PersonalityBench Informativeness indexChat & Writing27.733.4+5.8
AA-PersonalityBench Slop indexChat & Writing14.38.6−5.8
SAGEMath46.540.8−5.7
Financial Coaching BenchFinance & Accounting68.574.1+5.5
SnorkelUnderwrite 2.0Finance & Accounting25.030.5+5.5
Artificial Analysis Healthcare & Medical IndexIndexes52.046.6−5.4
Artificial Analysis Intelligence Index v4.3.2Indexes51.846.4−5.4
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine24.018.7−5.3
Vals Index v2.2Indexes63.958.6−5.3
Arena Alignment IndexSafety87.982.7−5.2
DAYJOB: AccountingFinance & Accounting25.220.0−5.2
AA-AnalystAgent (pass^5)Professional Work50.055.0+5.0
AA-Omniscience IndexIndexes41.532.0−4.7
Design Creativity BenchCode63.762.2−4.7
GDPxProfessional Work8.43.9−4.6
Artificial Analysis Finance & Accounting IndexIndexes56.251.7−4.5
APEX-Agents 1.1Professional Work73.268.7−4.5
FrontierMath ErdősMath4.40.0−4.4
DAYJOB: LawLegal31.827.6−4.2
AA-LCR v1.1Search84.080.0−4.0
CharXiv (descriptive and reasoning combined)Multimodal96.092.0−4.0
Vals Legal IndexIndexes68.772.7+4.0
KernelBench Mega Normalized (RTX PRO 6000)Code14.118.0+3.9
SWE Atlas v1.1 - OverallCode65.962.1−3.8
Excel Modeling BenchmarkFinance & Accounting70.867.0−3.8
Snorkel SWE-bench CLICode9.86.1−3.7
DAYJOB: FinanceFinance & Accounting18.014.5−3.5
LMArena Agent Arena Confirmed SuccessAgents11.75.1−3.3
Vals Finance IndexIndexes55.858.8+3.0
BioSimBench 1.0Science15.312.3−3.0
MedScribeHealth & Medicine86.589.4+2.9
Artificial Analysis Strategy & Ops IndexIndexes57.654.8−2.8
Vibe Code Bench v1.1Code88.986.2−2.8
HealthBenchHealth & Medicine56.954.2−2.7
Harvey HLAB — Criteria Pass RateLegal87.890.5+2.7
ProgramBenchCode3.00.5−2.5
HealthBench (length-adjusted, Artificial Analysis)Health & Medicine58.556.0−2.5
Ramp Accounting BenchFinance & Accounting46.444.1−2.3
Visa Agentic Commerce BenchmarkAgents62.260.0−2.2
AdvancedIFChat & Writing82.480.2−2.2
MMLU ProGeneral QA91.088.8−2.2
LMArena Agent Arena Bash ReliabilityAgents5.09.3+2.2
SciCode 1.0.1 (Artificial Analysis)Code55.857.8+2.0
KernelBench CUDA Normalized (RTX PRO 6000)Code43.245.2+1.9
OTIS Mock AIME 2024-2025Math100.098.1−1.9
Snorkel Digital Work IndexIndexes21.219.3−1.9
SAGE v1.1Math90.588.7−1.8
Political Science LLM BenchmarkProfessional Work68.366.6−1.8
MissionReportProfessional Work61.059.5−1.5
SWE Atlas - Codebase QnACode48.447.0−1.4
LMArena Agent Arena Bash RecoveryAgents4.57.2+1.3
Finance Agent v2.1Finance & Accounting61.860.6−1.2
ChessBench CoherenceGames87.586.3−1.2
AutomationBench-AA 1.0.6Agents66.665.6−1.0
ForecastBench Preliminary Dataset Brier IndexForecasting61.760.7−1.0
DAYJOB: TaxFinance & Accounting4.05.0+1.0
SpeciEval (Pooled English Runs)Safety97.496.4−1.0
SpeciEval (Refusal-Aware Retries, Historical)Safety97.496.4−1.0
DAYJOB: HealthcareHealth & Medicine7.68.4+0.8
SnorkelRevOpsProfessional Work12.111.3−0.8
MedCodeHealth & Medicine48.849.6+0.7
SWE-bench MultilingualCode84.984.2−0.7
ObviousBenchGeneral QA100.099.3−0.7
SWE-bench Multimodal (Vals v1, 480 Tasks)Code43.542.9−0.6
SAFE-Teen 106 worst branch: Relationships (under-18 prompt)Safety4.55.1−0.6
DTBenchGeneral QA96.596.0−0.5
Hermes BenchAgents73.974.4+0.4
Excel Modeling Benchmark v1.1Finance & Accounting71.171.5+0.4
Terminal-Bench 2.1Code85.485.0−0.4
GDP.xlsxProfessional Work22.022.3+0.3
Immigration BenchLegal67.868.0+0.2
Finance Agent v2Finance & Accounting52.052.3+0.2
CVDP (517-problem set)Code55.154.9−0.2
Snorkel WorkplaceAgentsProfessional Work15.015.2+0.2
SAFE-Teen Relationships (under-18 prompt)Safety0.91.1−0.2
SAFE-Teen 106: Relationships (under-18 prompt)Safety0.91.0−0.2
LMArena Agent Arena Tool HallucinationAgents1.01.00.0
AIME 2025Math100.0100.00.0
DAXBench v2Code92.692.60.0
Dual-Arm Sim (task completion)Agents0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal20.020.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.00.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal20.020.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0

GPT-6.1 SolvsGrok 4.7normalised score marginsData Nov 15, 2026Data: Vals AI, Artificial Analysis, Mercor and 59 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…