Atlas

Compare Models

Claude Opus 5 vs Grok 4.7

Claude Opus 5 leads by 3.5 capability points (≈95% interval +1.2 to +5.7): a clear lead over Grok 4.7 · P(higher) ≈ 100%

Wins 161 of 230 shared benchmarks (45 lost, 24 tied)

Capability vs. relative cost
Claude Opus 5Grok 4.7Closed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Opus 5vsGrok 4.7capability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins161 – 45 · 230 shared, 24 tied

Comparing matching setups only (10 benchmarks excluded) · suites counted once
← Claude Opus 5Grok 4.7 →

Claude Opus 5vsGrok 4.7wins by categoryData Nov 15, 2026Data: Vals AI, Mercor, Artificial Analysis and 64 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Grok 4.7 minus Claude Opus 5.

ProofBench v1.1Math99.026.0−73.0
FrontierMath Tier 4 (v2) - PrivateMath73.217.1−56.1
Medical Long Context Reasoning (MLCR-AA)Health & Medicine59.415.0−44.4
Furniture Assembly BenchmarkMultimodal60.820.8−40.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.040.0+40.0
BrokenArXiv 06/2026Math90.752.8−38.0
LLM Creative Story WritingChat & Writing3.70.4−37.3
LAB-Bench LitQA2 (SecureBio)Science51.416.9−34.5
CheatBenchSafety45.378.0−32.6
FrontierMath Tiers 1-3 (v2) - PrivateMath85.653.0−32.6
Toloka Arena (v3)Agents62.031.0−31.0
BioSecBench-Refusal V1 Clean RerunsSafety31.862.4+30.6
Mystery Game PuzzlesGames59.029.0−30.0
LAB-Bench ProtocolQA (SecureBio)Science81.151.4−29.7
ARC-AGI-2General QA90.461.4−29.0
ARC-AGI-3 (Semi-Private)General QA30.21.7−28.4
ALE-BenchCode26221823−28.3
HardGeoBench Avg DistanceMultimodal12182104−27.3
Bio Capabilities Index (SecureBio)Indexes178141−26.9
SWE-bench Verified Extended (Mercor held-out)Code82.055.3−26.7
IOI (Vals v2)Code84.357.7−26.6
Terminal-Bench 2.1 Extended (Mercor held-out)Code38.011.8−26.2
HPCT (SecureBio)Science78.653.7−24.9
SWE Atlas Codebase QnA Extended (Mercor held-out)Code40.015.3−24.7
FrontierSWE V2Code52.029.5−22.5
ChessBench CoherenceGames63.886.3+22.5
DeepsecBench — Revalidated V1Code32.410.9−21.5
DeepsecBench — Revalidated V1 (233 issues)Code32.310.9−21.4
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine40.018.7−21.3
Humanity's Last Exam DiamondGeneral QA44.823.6−21.2
Terminal-Bench 4.0 (Artificial Analysis)Code46.525.8−20.7
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal20.00.0−20.0
LAB-Bench CloningScenarios (SecureBio)Science100.080.1−19.9
LLM Chess EloGames1285822−19.0
HardGeoBench <=100miMultimodal46.327.5−18.7
LMArena Agent Arena Net ImprovementAgents9.54.0−18.2
DistressBenchSafety70.252.4−17.8
Extended NYT Connections 940 (Quadratic v1)Games94.376.8−17.5
BuseyBench SVGMultimodal6.34.6−17.3
Terminal-Bench Science 0.1 (Vals)Science27.110.0−17.1
Terminal-Bench-Science 0.1Science28.611.4−17.1
Arena Image-to-WebDevCode16641589−16.4
Terminal-Bench 4.0Code53.937.6−16.4
Conceptual Reasoning IndexIndexes74.763.1−15.9
LMCAGeneral QA64.549.4−15.1
HardGeoBench <=10miMultimodal32.517.5−15.0
ChessBench EloGames8261076+14.9
LLM Debate BenchmarkChat & Writing17241620−14.8
MBCT (SecureBio)Science60.246.1−14.1
MedCodeHealth & Medicine63.649.6−14.0
Toloka Arena (v1)Agents52.939.3−13.6
Vals Science IndexIndexes47.934.8−13.1
ScBench-Long V1 Clean RerunsScience24.411.4−13.1
HealthBenchHealth & Medicine67.154.2−12.9
Vals RSI Index v1.1 (four tasks)Agents33.020.2−12.8
Code MigrationCode57.544.8−12.7
GameBench 2Games84.872.3−12.5
Physical-Engineering's Last Exam (hidden test)Professional Work42.630.2−12.4
SnorkelUnderwrite 2.0Finance & Accounting18.330.5+12.2
MysteryMechanismScience37.425.2−12.2
MedXpertQA MM Extended (Mercor held-out)Health & Medicine64.152.0−12.1
GeneBench-Pro Public Release SubsetScience22.010.0−12.0
TaxCalcBench TY25Finance & Accounting20.08.0−12.0
GBENCH One-shot Coding GScoreGames68.156.3−11.9
Ruby LLM Benchmarks — Program FixerCode79.868.0−11.9
ComplexConstraintsChat & Writing37.348.9+11.6
τ³-Banking (Mercor run, bm25_grep retrieval)Agents44.032.6−11.4
Public Benefits Bench v1.1Professional Work76.965.6−11.3
Humanity's Last Exam (Text-Only)General QA54.443.1−11.3
dfbench Detection RecallCybersecurity45.456.6+11.2
WCB (SecureBio)Science68.857.6−11.2
CritPt (avg@5)Science29.118.0−11.1
Eval Connections One-Shot with TrapsGames74.085.0+11.0
Olam Social Poker EloGames15181508−10.9
MedXpertQA MM (200-question set)Health & Medicine79.869.0−10.8
PLE (Physical-Engineering's Last Exam)Professional Work42.632.0−10.6
APEX-SWECode63.753.4−10.3
BioMysteryBench (Vals Terminus 2)Science79.369.3−10.0
HardGeoBench Country AccuracyMultimodal53.843.8−10.0
ACCoRDGeneral QA83.874.0−9.8
ReactBench (39 tasks)Code42.132.3−9.7
MineBench — Bradley-Terry RatingGames20081848−9.7
Vision Arena OverallMultimodal12851249−9.4
CritPt v2 (agentic, avg@5)Science71.462.3−9.1
AutomationBench-AA 1.0.6Agents56.665.6+9.0
LMArena Agent Arena Net Improvement (Steering Burden)Agents7.22.6−8.8
BrowseComp (Mercor 130-question subset)Search84.675.8−8.8
Artificial Analysis Intelligence Index v5.0.1Indexes44.235.5−8.7
SAGEMath49.440.8−8.6
YC-BenchGames19115921666389−8.6
SpeciEval (Pooled English Runs)Safety87.996.4+8.5
SpeciEval (Refusal-Aware Retries, Historical)Safety87.996.4+8.5
Design Arena Overall Code CategoriesCode13211270−8.5
LMArena Text Arena EloChat & Writing14901442−8.3
APEX-Agents 1.1Professional Work77.068.7−8.3
Legal Research BenchLegal55.347.1−8.2
CadQueryEval (Corrected Geometry)Code80.088.0+8.0
LiveBench 2.0Agents82.174.1−8.0
RSI-Exam 0.1 (Full 88)Agents0.50.4−8.0
Snorkel SWE-bench CLICode14.06.1−7.9
APEX-AccountingFinance & Accounting54.546.7−7.8
Tax Agent BenchFinance & Accounting76.769.0−7.7
Roboflow Vision Evals (October 2026, five tasks)Multimodal70.963.5−7.4
VariantBench V1 Clean RerunsScience52.845.5−7.3
ARC-AGI-1 Semi-PrivateGeneral QA97.590.2−7.3
ATLAS-Finance v1.0Finance & Accounting12.35.0−7.3
CVDP (517-problem set)Code62.254.9−7.3
Senior SWE-BenchCode34.727.4−7.3
Arena Alignment IndexSafety75.682.7+7.1
DeepResearch Bench II (Mercor 130-task set)Search56.149.0−7.1
Snorkel Agentic Coding 2.0Code38.931.9−7.0
Epoch Capabilities Index (ECI)Indexes164156−6.8
Excel Modeling BenchmarkFinance & Accounting73.667.0−6.6
SpatialBench-Long V1 Clean RerunsScience30.924.4−6.5
LMArena Code Arena WebDevCode16911641−6.5
SnorkelLegalLegal26.432.8+6.4
Roboflow Vision Evals (September 2026 protocol)Multimodal78.371.9−6.4
DAXBench v2Code86.292.6+6.4
Creative Writing v3 EloChat & Writing21352012−6.2
VIALSMultimodal24.418.2−6.2
MMMU ProMultimodal84.078.2−5.8
AA-LCR Extended (Mercor held-out)Search73.968.2−5.7
MetagenomicsBench V0Science54.048.3−5.7
AutoResearchExam (hidden-test AUARC)Agents0.60.5−5.6
RedlineBenchLegal44.049.5+5.5
FrontierPhysics v0.1Science9.54.2−5.3
TxBench Oligo Discovery V1 Clean RerunsScience45.640.3−5.3
LMArena Agent Arena SteerabilityAgents10.70.5−5.1
TxBench Preclinical Pharmacology V1 Clean RerunsScience59.754.7−5.0
GPQA DiamondGeneral QA87.992.7+4.9
MetagenomicsBench V1Science53.348.7−4.7
RuneBenchGames5.75.4−4.6
LMArena Agent Arena Praise vs ComplaintAgents12.23.1−4.5
RCM-Bench-1Health & Medicine92.788.2−4.5
BrowseComp Extended (Mercor held-out)Search44.039.5−4.5
BioSecBench-Surveillance V1 Clean RerunsScience43.648.0+4.4
FrontierCode 1.1 Extended - Pass RateCode69.665.2−4.4
Artificial Analysis Intelligence Index v4.3.2Indexes50.846.4−4.3
ProgramBench (Mercor 100-task subset)Code4.30.0−4.3
Terminal-Bench 2.1Code89.185.0−4.1
ABC-Bench liquid handling (SecureBio)Science94.098.0+4.0
Next.js AI Agent EvaluationsAgents100.096.0−4.0
Vending-Bench 2Games1118210537−3.9
Artificial Analysis Progress Index v1Indexes334323−3.9
Clipping Bench v1.0Games734384−3.8
SnorkelFinance 2.0Finance & Accounting16.520.2+3.7
Artificial Analysis Coding Agent Index v1.5Indexes59.756.3−3.5
LMArena Agent Arena Steering BurdenAgents4.4-2.3−3.3
Longform Creative WritingChat & Writing86.383.0−3.3
ChessBench AccuracyGames71.074.0+3.2
MMLU ProGeneral QA92.088.8−3.2
DAYJOB: FinanceFinance & Accounting11.314.5+3.2
APEX-1 (Mercor, mean score)Professional Work68.665.4−3.2
Biopharma Bench V0.1Health & Medicine67.664.4−3.2
LMArena Agent Arena Bash RecoveryAgents13.67.2−3.2
SWE-Marathon v1.1Code48.845.6−3.1
Realm Financial ReasoningFinance & Accounting47.444.5−2.9
Harvey HLAB — Criteria Pass RateLegal87.790.5+2.8
SnorkelManufacturingProfessional Work9.412.1+2.7
CharXiv (descriptive and reasoning combined)Multimodal94.692.0−2.6
LegalBenchLegal87.084.4−2.6
LMArena Agent Arena Confirmed SuccessAgents10.25.1−2.6
AA-Omniscience IndexIndexes37.132.0−2.5
Snorkel Digital Work IndexIndexes21.819.3−2.4
Vibe Code Bench v1.1Code88.486.2−2.2
TxBench Antibody Discovery V1 Clean RerunsScience40.642.8+2.2
DeepSWE v1.1Code73.271.0−2.2
Blueprint-Bench 2Multimodal0.30.3+2.1
AA-LCR v1.1Search82.080.0−2.0
Chess Puzzles (task version 2.0)Games43.041.0−2.0
Bug Hunt BenchCode25.727.6+1.9
DTBenchGeneral QA97.996.0−1.9
MedScribeHealth & Medicine91.089.4−1.6
CharXiv Extended (Mercor held-out)Multimodal79.878.2−1.6
Concept Synth INDUCTION Challenge64 Round 1General QA32.834.4+1.6
Creative Writing v3 Rubric ScoreChat & Writing16.315.9−1.6
BioSecBench-Function V1 Clean RerunsScience44.843.3−1.4
SciCode 1.0.1 (Artificial Analysis)Code56.457.8+1.4
ForecastBench Preliminary Dataset Brier IndexForecasting62.060.7−1.3
SWE-TogetherCode63.865.1+1.3
AA-AnalystAgent (pass^5)Professional Work53.855.0+1.3
GDP.pdfProfessional Work24.022.8−1.2
SpatialBench V1 Clean RerunsScience68.469.6+1.2
DeepsecBench-AA (Artificial Analysis)Cybersecurity25.926.9+1.1
GDPval-AA v2.1Professional Work16951717+1.0
FrontierCode 2.0 - Pass RateCode31.432.3+0.9
SnorkelRevOpsProfessional Work12.211.3−0.9
ScBench V1 Clean RerunsScience57.658.5+0.9
OTIS Mock AIME 2024-2025Math98.998.1−0.8
FrontierCode 2.0 - ScoreCode28.128.9+0.8
ObviousBenchGeneral QA100.099.3−0.7
SM BenchSafety85.886.3+0.5
EnterpriseBench: CoreCraftProfessional Work68.768.2−0.5
AA-Briefcase v1.1 EloAgents16291636+0.3
CursorBench 4.0Code46.646.3−0.3
Snorkel WorkplaceAgentsProfessional Work14.915.2+0.3
AdvancedIFChat & Writing80.580.2−0.3
KernelBench CUDA Normalized (RTX PRO 6000)Code45.545.2−0.3
LMArena Agent Arena Bash ReliabilityAgents9.89.3−0.2
GPQA-Bio (SecureBio)Science82.782.5−0.2
MMLU-Bio (SecureBio)Science91.190.9−0.2
LMArena Agent Arena Tool HallucinationAgents0.91.00.0
Aikido Known-CVE Detection (32 CVEs) single-run averageCybersecurity70.870.80.0
AIME 2025Math100.0100.00.0
Concept Synth INDUCTION Challenge100 Round 1General QA24.024.00.0
DAYJOB: HealthcareHealth & Medicine8.48.40.0
EpiBench V1 Clean RerunsScience20.820.80.0
FrontierMath ErdősMath0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal20.020.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.00.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal20.020.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal20.020.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0

Claude Opus 5vsGrok 4.7normalised score marginsData Nov 15, 2026Data: Vals AI, Mercor, Artificial Analysis and 64 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…