Atlas

Compare Models

Opus 4.8 vs Grok 4.6

Grok 4.6 leads by 0.3 capability points (≈95% interval −1.7 to +2.2): statistically tied · P(higher) ≈ 61%

Opus 4.8 wins 113 of 241 shared benchmarks (107 lost, 21 tied)

Capability vs. relative cost
Opus 4.8Grok 4.6Closed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Opus 4.8vsGrok 4.6capability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins113 – 107 · 241 shared, 21 tied

Comparing matching setups only (21 benchmarks excluded) · suites counted once
← Opus 4.8Grok 4.6 →

Opus 4.8vsGrok 4.6wins by categoryData Nov 15, 2026Data: Vals AI, Mercor, benchmarks.bio and 66 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Grok 4.6 minus Opus 4.8.

Gray Swan IPI Attack Success (k=15)Safety8.051.8−43.8
LLM Creative Story WritingChat & Writing0.8-2.9−41.1
KernelBench Mega Normalized (RTX PRO 6000)Code40.60.0−40.6
NanoGPT Speedrun Frontier (human record gap closed)Agents39.410.1−39.3
BioSecBench-RefusalSafety28.664.2+35.6
Medical Long Context Reasoning (MLCR-AA)Health & Medicine45.612.2−33.3
Document ArenaProfessional Work14751438−32.6
VoxelBench Text RatingGames16272050+30.1
Concept Synth INDUCTION Challenge64 Round 1General QA6.335.9+29.7
KernelBench Hard Normalized (RTX PRO 6000)Code32.660.5+27.8
SAGEMath54.828.9−25.9
FrontierMath Tier 4 (v2) - PrivateMath56.131.7−24.4
Antidote: Everyday EditionSafety1030966−22.9
MRCR v2 (8-needle, Context Arena) - 128K pointwiseGeneral QA75.252.6−22.5
AutomationBench-AA 1.0.6Agents45.667.0+21.4
Olam Social Poker EloGames15181499−20.7
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal20.040.0+20.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal40.020.0−20.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.020.0+20.0
Wiz Cyber Model Arena v3.9Cybersecurity65.445.5−19.9
LAB-Bench LitQA2 (SecureBio)Science48.728.9−19.8
Bio Capabilities Index (SecureBio)Indexes164137−19.4
Chess Puzzles (task version 2.0)Games52.033.0−19.0
LMArena Agent Arena Net ImprovementAgents6.91.1−18.9
Blueprint-Bench 2Multimodal0.10.3+18.7
HardGeoBench Avg DistanceMultimodal12451807−17.3
AHK-EvalCode80.697.2+16.7
𝜏³-BankingAgents34.250.7+16.5
WCB (SecureBio)Science66.750.3−16.4
SM BenchSafety72.989.0+16.1
WeirdML (v2)Science82.967.3−15.6
Hemingway-benchChat & Writing1038983−15.2
SWE-bench Verified Extended (Mercor held-out)Code63.378.0+14.7
FrontierMath Tiers 1-3 (v2) - PrivateMath80.066.0−14.0
APEX-SWECode43.957.7+13.8
MRCR v2 (8-needle, Context Arena) - 128K AUCGeneral QA87.874.4−13.4
EnterpriseBench: CoreCraftProfessional Work52.365.6+13.3
Bug Hunt BenchCode14.327.3+13.0
AA-Briefcase EloAgents13541577+13.0
APEX-Agents 1.1Professional Work64.477.3+12.9
Terminal-Bench 2.1 Extended (Mercor held-out)Code32.745.5+12.8
KernelBench CUDA Normalized (RTX PRO 6000)Code24.937.7+12.8
DeepSWE v1.1Code54.466.7+12.4
HealthBench ProfessionalHealth & Medicine60.348.5−11.8
RuneBenchGames5.15.8+11.5
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting66.477.8+11.3
TxBench Antibody Discovery V1 Clean RerunsScience32.443.5+11.2
SimpleBenchGeneral QA64.875.9+11.1
Extended NYT Connections 940 (Quadratic v1)Games91.180.0−11.1
GDPval-AA v2Professional Work15291753+11.0
ChessBench CoherenceGames79.769.2−10.5
BrowseComp (Mercor 130-question subset)Search73.784.0+10.3
HardGeoBench Country AccuracyMultimodal53.843.8−10.0
ALE-BenchCode21021822−9.9
DeepsecBench — Revalidated V1Code6.916.2+9.3
DeepsecBench — Revalidated V1 (233 issues)Code6.916.1+9.2
LMCAGeneral QA57.548.5−9.1
LAB-Bench ProtocolQA (SecureBio)Science76.968.1−8.8
RedlineBenchLegal39.748.5+8.8
GDPval-AA v2.1Professional Work14611658+8.7
MedCodeHealth & Medicine53.244.7−8.5
BrowseComp Extended (Mercor held-out)Search30.021.5−8.5
SpatialBench V0 VerifiedScience69.077.4+8.4
HLE-Bio (SecureBio)Science37.629.2−8.4
NoCoT-Bench NCRI 15.2General QA10286.8−8.4
LMArena Code Arena WebDevCode15551618+8.2
GlobeBenchGeneral QA20.028.1+8.1
TxBench Oligo Discovery V1 Clean RerunsScience35.343.3+8.0
MetagenomicsBench V0Science50.042.0−8.0
MetagenomicsBench V1Science50.042.0−8.0
Conceptual Reasoning IndexIndexes68.362.5−8.0
PWNBench v0.1Cybersecurity55.663.3+7.7
CursorBench 3.2Code62.369.9+7.6
HardGeoBench <=10miMultimodal27.520.0−7.5
AdvancedIFChat & Writing73.780.9+7.2
Terminal-Bench Science 0.1 (Vals)Science4.311.4+7.1
SpatialBench-Long V1 Clean RerunsScience33.326.2−7.1
SWE-bench VerifiedCode88.695.6+7.0
Co-Design BenchCode58.151.1−7.0
GDP.pdfProfessional Work24.017.2−6.8
Excel Modeling BenchmarkFinance & Accounting69.462.7−6.6
SpeciEval (Pooled English Runs)Safety84.390.8+6.5
SpeciEval (Refusal-Aware Retries, Historical)Safety84.390.8+6.5
GPQA-Bio (SecureBio)Science80.987.4+6.5
Vibe Code Bench v1.1Code82.776.2−6.5
Tax Agent BenchFinance & Accounting64.971.4+6.5
Terminal-Bench 2.1Code82.088.4+6.4
HieroglyphBenchMultimodal14.020.3+6.3
GameBench 2Games62.568.7+6.3
Arena Alignment IndexSafety74.580.8+6.2
TxBench Antibody Discovery V0Science45.651.9+6.2
VariantBenchScience39.846.0+6.2
BuseyBench SVGMultimodal5.24.6−6.0
Time Horizon Index: KSPAgents11.85.8−6.0
xbench-ScienceQAScience76.470.4−6.0
MBCT (SecureBio)Science56.450.4−6.0
Design Arena Overall Code CategoriesCode12521288+6.0
SWE Atlas - Codebase QnACode44.450.3+5.9
Vision Arena OverallMultimodal12861263−5.8
ACCoRDGeneral QA77.471.6−5.8
MortgageTax (Vals all 1,258 documents)Finance & Accounting69.964.2−5.7
SkillsBench without SkillsAgents43.738.2−5.5
SUPERChemScience60.365.7+5.4
HANDBOOK.mdProfessional Work21.927.3+5.4
MMMU-Pro Extended (Mercor held-out)Multimodal31.426.0−5.4
ScBench-Long V1 Clean RerunsScience21.015.9−5.1
APEX-1 (Mercor, mean score)Professional Work62.767.7+5.0
ARC-AGI-1 Semi-PrivateGeneral QA92.587.5−5.0
WordleBenchGames94.089.0−5.0
YC-BenchGames17113891568241−5.0
ARC-AGI-2General QA72.167.1−5.0
Finance Agent v2 - Market AnalysisFinance & Accounting67.262.2−5.0
ChessBench AccuracyGames72.467.8−4.9
SCI-AGENT-EXAMScience57.152.4−4.7
SWE Atlas Codebase QnA Extended (Mercor held-out)Code24.019.3−4.7
Harvey HLAB — Criteria Pass RateLegal87.992.5+4.7
Toloka Arena (v1)Agents54.349.7−4.6
TxBench Oligo Discovery V0Science39.444.0+4.6
TaxEval v2Finance & Accounting75.671.1−4.5
Finance Agent v2 - Financial ModelingFinance & Accounting23.318.9−4.4
LMArena Text Arena EloChat & Writing14821456−4.4
EnterpriseOps-Gym-AAAgents44.048.3+4.4
Legal Research BenchLegal43.848.1+4.3
Senior SWE-BenchCode30.526.3−4.2
ChessBench EloGames961891−4.2
CadQueryEval (Corrected Geometry)Code68.072.0+4.0
TxBench Preclinical Pharmacology V1 Clean RerunsScience57.761.7+4.0
LLM Debate BenchmarkChat & Writing16311604−3.8
NoCoT-Bench NCKI (kspine v3)General QA117112−3.8
AA-AnalystAgent (pass^5)Professional Work45.041.3−3.7
HardGeoBench <=100miMultimodal36.332.5−3.7
Finance Agent v2 - PrecedentsFinance & Accounting25.621.9−3.7
SimpleQA Verified (Epoch anti-abstention)General QA53.049.3−3.7
Finance Agent v2 - ComparablesFinance & Accounting44.941.2−3.6
CorpFin v2 - Max Fitting ContextFinance & Accounting69.866.2−3.6
HLE Extended (Mercor held-out)General QA49.445.8−3.6
GPQA DiamondGeneral QA91.995.5+3.5
SkillsBench with SkillsAgents59.255.8−3.5
MedXpertQA MM Extended (Mercor held-out)Health & Medicine51.648.4−3.2
LMArena Agent Arena SteerabilityAgents9.63.3−3.2
Vals Index v2.1Indexes55.152.1−3.0
GeneBench-Pro Public Release SubsetScience18.015.0−3.0
GDPval Extended (Mercor held-out rubric score)Professional Work58.961.8+2.9
LegalBenchLegal83.686.3+2.7
MedXpertQA MM (200-question set)Health & Medicine72.575.2+2.7
Code MigrationCode47.344.6−2.7
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine16.018.7+2.7
BioSecBench-Function V0Science44.642.0−2.6
Artificial Analysis Intelligence Index v4.3.2Indexes41.844.3+2.5
Furniture Assembly BenchmarkMultimodal42.540.0−2.5
Artificial Analysis Intelligence Index v4.3Indexes42.044.4+2.4
DTBenchGeneral QA94.997.3+2.4
SciCode (main problems)Science16.914.6−2.3
EpiBench V1 Clean RerunsScience25.823.6−2.2
SciCode 1.0.1 (Artificial Analysis)Code54.456.5+2.1
ScBench V1 Clean RerunsScience57.159.1+2.1
AA-LCR Extended (Mercor held-out)Search63.265.2+2.0
Mystery Game PuzzlesGames36.034.0−2.0
VariantBench V1 Clean RerunsScience42.440.4−2.0
EBR-bench v3Games28.630.5+1.9
Humanity's Last Exam (Text-Only)General QA44.842.9−1.9
CorpFin v2 - Shared Max ContextFinance & Accounting66.067.8+1.9
LiveBench 2.0Agents76.874.9−1.8
Finance Agent v2 - Earnings AnalysisFinance & Accounting72.173.9+1.8
Vals Index v2Indexes60.959.2−1.7
FrontierCode 1.1 Extended - ScoreCode59.661.3+1.7
AIME 2025Math98.3100.0+1.7
LAB-Bench CloningScenarios (SecureBio)Science95.994.2−1.7
Vals RSI Index v1.1 (historical five tasks)Agents23.221.5−1.7
Terminal-Bench 4.0Code23.625.3+1.7
Epoch Capabilities Index (ECI)Indexes160158−1.6
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting71.673.2+1.6
Humanity's Last Exam DiamondGeneral QA23.121.5−1.6
FrontierCode 1.1 Extended - Pass RateCode65.567.0+1.6
ReactBench (39 tasks)Code32.333.8+1.5
PG-LLM-50Science0.40.3−1.5
ScBench-Long V0 VerifiedScience25.824.2−1.5
MineBench — Bradley-Terry RatingGames17861811+1.5
SpatialBench-Long V0 VerifiedScience28.830.3+1.5
BioSecBench-SurveillanceScience50.251.7+1.5
GDPval (Mercor 213-task rubric score)Professional Work84.586.0+1.5
HPCT (SecureBio)Science63.161.6−1.5
Vals RSI Index v1.1 (four tasks)Agents24.025.5+1.5
Finance Agent v2 - Disclosure AnalysisFinance & Accounting65.764.4−1.4
GBENCH One-shot Coding GScoreGames54.055.4+1.3
Public Benefits Bench v1.1Professional Work68.166.8−1.3
Finance Agent v2 - AdjustmentsFinance & Accounting48.449.6+1.2
Artificial Analysis Progress Index v1Indexes318322+1.1
BioSecBench-Function V1 Clean RerunsScience43.144.0+1.0
ComplexConstraintsChat & Writing35.636.5+0.9
SWE-TogetherCode61.560.6−0.9
MMLU-Bio (SecureBio)Science90.991.8+0.9
SpatialBench V1 Clean RerunsScience67.066.1−0.9
AA-Omniscience IndexIndexes28.830.5+0.9
OTIS Mock AIME 2024-2025Math98.399.2+0.8
Roboflow Vision Evals (October 2026, five tasks)Multimodal58.359.1+0.8
LiveCodeBench (Vals Public) - MediumCode89.889.0−0.8
MedScribeHealth & Medicine85.886.5+0.8
LMArena Agent Arena Bash RecoveryAgents7.25.7−0.8
ObviousBenchGeneral QA99.3100.0+0.7
BioSecBench-Surveillance V1 Clean RerunsScience46.947.5+0.6
Finance Agent v2 - All PassFinance & Accounting43.542.9−0.6
ForecastBench Preliminary Dataset Brier IndexForecasting61.661.0−0.6
CharXiv Extended (Mercor held-out)Multimodal76.777.3+0.6
MMLU ProGeneral QA90.089.4−0.6
ARC-AGI-3 (Semi-Private)General QA1.52.1+0.6
CyberArenaCybersecurity65.065.5+0.5
Terminal-Bench 4.0 (Artificial Analysis)Code21.721.2−0.5
ForecastBench Tournament Overall Brier IndexForecasting66.265.8−0.4
CharXiv (descriptive and reasoning combined)Multimodal93.292.9−0.3
LLM Chess EloGames450443−0.3
CorpFin v2 - Exact PagesFinance & Accounting64.364.5+0.1
VCT (SecureBio)Science48.248.3+0.1
BabaIsBench Stage 0: The IntroGames96.096.00.0
BlenderWorldCode57.057.00.0
EpiBenchScience37.437.40.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal20.020.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal20.020.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
LiveCodeBench (Vals Public) - EasyCode99.199.10.0
ProgramBenchCode1.01.00.0
Roboflow Vision Evals (September 2026 protocol)Multimodal68.768.70.0
ZeroBench (Main questions, pass@5)Multimodal17.017.00.0

Opus 4.8vsGrok 4.6normalised score marginsData Nov 15, 2026Data: Vals AI, Mercor, benchmarks.bio and 66 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…