Atlas

Compare Models

Opus 4.8 vs GPT-6 Astra

GPT-6 Astra leads by 7.5 capability points (≈95% interval +5.2 to +9.8): a clear lead over Opus 4.8 · P(higher) ≈ 100%

Wins 193 of 253 shared benchmarks (43 lost, 17 tied)

Capability vs. relative cost
Opus 4.8GPT-6 AstraClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Opus 4.8vsGPT-6 Astracapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins43 – 193 · 253 shared, 17 tied

Comparing matching setups only (97 benchmarks excluded) · suites counted once
← Opus 4.8GPT-6 Astra →

Opus 4.8vsGPT-6 Astrawins by categoryData Nov 15, 2026Data: Vals AI, benchmarks.bio, Mercor and 70 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6 Astra minus Opus 4.8.

Clocktower BenchGames11951892+100.0
Concept Synth INDUCTION Challenge64 Round 1General QA6.398.4+92.2
Time Horizon Index: KSPAgents11.890.5+78.7
ActiveVisionMultimodal2.476.5+74.1
VoxelBench Text RatingGames16272625+71.1
BrokenArXiv 05/2026Math23.091.0+68.0
BrokenArXiv 04/2026Math34.497.5+63.1
BrokenArXiv 06/2026Math41.799.1+57.4
Terminal-Bench Science 0.1 (Vals)Science10.065.7+55.7
ARC-AGI-3 (Semi-Private)General QA1.554.8+53.3
EnigmaEvalGeneral QA20.569.7+49.2
Mystery Game PuzzlesGames36.084.0+48.0
LLM Chess EloGames4501614+47.8
ChessBench EloGames9611755+47.2
ChessBench — Historical Output RepairGames24.066.0+42.0
FrontierMath Tier 4 (v2) - PrivateMath56.197.6+41.5
Humanity's Last Exam DiamondGeneral QA23.163.8+40.7
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal40.00.0−40.0
AutoformBenchMath44.082.4+38.4
Terminal-Bench 4.0 (Artificial Analysis)Code21.759.6+37.9
ALE-BenchCode21023164+37.6
Furniture Assembly BenchmarkMultimodal42.580.0+37.5
Terminal-Bench 4.0Code23.259.6+36.4
Blueprint-Bench 2Multimodal0.10.5+35.2
HPCT Refusal (CAIS)Safety85.050.0−35.0
ZeroBench (Main questions, pass@5)Multimodal17.052.0+35.0
Olam Social Poker EloGames15181550+34.8
TextQuests Game ProgressGames40.174.2+34.1
Agent Red Teaming k=100 (CAIS)Safety47.013.5+33.5
TasteVal (compute multiplier)Science0.81.6+33.4
Into the Unknown Integrity Node Pass RateScience65.397.8+32.5
GSO Opt@1Code47.179.4+32.3
CadQueryEval (Corrected Geometry)Code68.0100.0+32.0
GameBench 2Games62.594.3+31.9
NoCoT-Bench NCRI 15.2General QA102159+31.3
DeepsecBench — Revalidated V1Code6.937.8+30.9
DeepsecBench — Revalidated V1 (233 issues)Code6.937.6+30.8
LLM Creative Story WritingChat & Writing0.83.4+30.3
Humanity's Sixth SenseMultimodal23.553.6+30.1
ArXivMath 05/2026Math65.095.0+30.0
HiL-Bench (Human-in-Loop Benchmark)Agents35.365.3+30.0
GlobeBenchGeneral QA20.049.6+29.5
MMMU-Pro Extended (Mercor held-out)Multimodal31.460.0+28.6
ArXivMath 04/2026Math62.590.8+28.3
SpatialViz-BenchMultimodal65.293.1+27.9
TextQuests Average Cumulative HarmSafety21.029.2−27.9
MindCube TinyMultimodal64.992.7+27.8
HieroglyphBenchMultimodal14.040.0+26.0
GBENCH One-shot Coding GScoreGames54.079.2+25.2
Roboflow Vision Evals (October 2026, five tasks)Multimodal58.383.4+25.1
BullshitBench v2 LeaderboardSafety94.069.0−25.0
RiemannBenchMath47.272.0+24.8
AutomationBench 1.0.6 (Zapier)Agents16.941.4+24.5
ArXivMath 06/2026Math70.094.4+24.5
TaxCalcBench TY25Finance & Accounting20.044.0+24.0
SnakeBench Average ApplesGames15.119.8+23.9
SCI-AGENT-EXAMScience57.180.9+23.8
VCT-Refusal (CAIS)Safety41.217.6−23.6
Into the Unknown Direction Node Pass RateScience60.183.6+23.5
ABC-Bench liquid handling (SecureBio)Science100.076.7−23.3
RedlineBenchLegal39.762.8+23.1
ARC-AGI-2General QA72.195.0+22.9
AutomationBench-AA 1.0.6Agents45.668.5+22.9
ERQAMultimodal59.582.3+22.8
SimpleQA Verified (Epoch anti-abstention)General QA53.075.6+22.6
Machiavelli Deception (CAIS)Safety83.776.3+22.5
IntPhys 2Multimodal55.177.4+22.3
FACTS Benchmark SuiteGeneral QA53.575.7+22.2
ComplexConstraintsChat & Writing35.657.7+22.1
Chess Puzzles (task version 2.0)Games52.074.0+22.0
DrugDiscoveryBenchScience46.868.7+21.9
BuseyBench SVGMultimodal5.27.3+21.1
BrowseComp (Mercor 130-question subset)Search73.794.2+20.5
Code MigrationCode47.367.7+20.5
LVBenchMultimodal67.387.5+20.2
Harvey's Legal Agent Benchmark - Capital MarketsLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.020.0+20.0
SnakeBench Win RateGames60.080.0+20.0
SUPERChemScience60.380.3+20.0
DeepSWE v1.1Code54.474.1+19.8
AHK-EvalCode80.6100.0+19.4
SimpleBenchGeneral QA64.883.6+18.8
TxBench Antibody Discovery V1 Clean RerunsScience30.248.9+18.7
GeneBench-Pro Public Release SubsetScience18.036.0+18.0
Roboflow Vision Evals (September 2026 protocol)Multimodal68.786.6+17.9
Agents' Last Exam (ALE)Agents16.434.2+17.8
ChessBench CoherenceGames79.797.0+17.3
ScBench-Long V0 VerifiedScience25.842.4+16.7
PCT Political Inconsistency (CAIS)Safety53.837.4+16.4
TxBench Oligo Discovery V0Science39.455.5+16.1
BioSecBench-RefusalSafety28.644.6+16.0
Terminal-Bench 2.1Code71.987.3+15.4
HardGeoBench <=100miMultimodal36.351.3+15.0
BioTIER RefusalSafety94.779.9−14.8
CRED v0.92Science85.099.5+14.5
AA-Briefcase EloAgents13161562+14.3
MedXpertQA MM (200-question set)Health & Medicine72.586.8+14.3
TxBench Oligo Discovery V1 Clean RerunsScience35.349.2+13.9
FrontierMath Tiers 1-3 (v2) - PrivateMath80.093.7+13.7
ChessBench AccuracyGames72.485.0+13.5
Arena Alignment IndexSafety74.587.8+13.3
Drone-BenchAgents82.395.1+12.8
Remote Labor Index (RLI)Professional Work8.320.8+12.5
LAB-Bench ProtocolQA (SecureBio)Science76.989.3+12.4
SpeciEval (Pooled English Runs)Safety84.396.5+12.2
SpeciEval (Refusal-Aware Retries, Historical)Safety84.396.5+12.2
VCT v2 (SecureBio)Science45.958.0+12.1
Co-Design BenchCode58.169.9+11.8
LMArena Agent Arena Net ImprovementAgents6.910.4+11.5
TxBench Antibody Discovery V0Science45.656.9+11.3
Artificial Analysis Progress Index v1Indexes318350+11.0
Artificial Analysis Intelligence Index v4.3.2Indexes41.852.7+10.9
Artificial Analysis Intelligence Index v4.3Indexes42.052.8+10.8
ScBench-Long V1 Clean RerunsScience16.527.3+10.8
APEX-Agents 1.1Professional Work64.475.1+10.7
Medical Long Context Reasoning (MLCR-AA)Health & Medicine45.635.0−10.6
ScreenSpot-ProMultimodal82.392.7+10.4
HardGeoBench Avg DistanceMultimodal1245908+10.4
WeirdML (v2)Science82.993.3+10.4
VCT (SecureBio)Science48.258.5+10.3
HardGeoBench <=10miMultimodal27.537.5+10.0
APEX-AccountingFinance & Accounting48.057.9+9.9
EnterpriseBench: CoreCraftProfessional Work52.342.6−9.7
NoCoT-Bench NCKI (kspine v3)General QA117131+9.6
GDP.pdf (Artificial Analysis, All-pass)Professional Work22.832.2+9.4
HealthBench ProfessionalHealth & Medicine60.369.7+9.4
LiveBench 2.0Agents76.886.1+9.4
MedXpertQA MM Extended (Mercor held-out)Health & Medicine51.660.8+9.2
𝜏³-BankingAgents34.243.1+8.9
SAGEMath54.846.4−8.4
CharXiv Extended (Mercor held-out)Multimodal76.785.1+8.4
AdvancedIFChat & Writing73.782.0+8.3
Conceptual Reasoning IndexIndexes68.373.9+7.7
MBCT (SecureBio)Science56.463.9+7.5
AA-Omniscience IndexIndexes28.843.7+7.5
Epoch Capabilities Index (ECI)Indexes160169+7.5
GPQA DiamondGeneral QA88.495.8+7.4
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting71.678.8+7.1
Finance Agent v2 - ComparablesFinance & Accounting44.937.8−7.1
Extended NYT Connections 940 (Quadratic v1)Games91.198.1+7.0
LMCAGeneral QA57.564.4+6.9
ScBench V1 Clean RerunsScience57.163.9+6.8
SnakeBench RatingGames35.437.9+6.8
APEX-1 (Mercor, mean score)Professional Work62.769.3+6.6
Document ArenaProfessional Work14751467−6.5
GPQA-Bio (SecureBio)Science80.987.4+6.5
VariantBenchScience42.148.6+6.5
LisanBench Path LengthGames94637393−6.4
Senior SWE-BenchCode30.536.8+6.3
HardGeoBench Country AccuracyMultimodal53.860.0+6.3
AA-AnalystAgent (pass^5)Professional Work45.051.3+6.2
LAB-Bench LitQA2 (SecureBio)Science48.754.8+6.1
SpatialBench-Long V0 VerifiedScience28.834.9+6.1
ARC-AGI-1 Semi-PrivateGeneral QA92.598.5+6.0
SpatialBench-Long V1 Clean RerunsScience33.339.3+6.0
HPCT (SecureBio)Science63.169.0+5.9
Vals Index v2Indexes60.966.6+5.7
EpiBenchScience39.044.6+5.7
BioSecBench-Function V1 Clean RerunsScience43.148.6+5.6
GDPval-AA v2.1Professional Work14611587+5.5
DeepResearch Bench II (Mercor 130-task set)Search44.650.1+5.5
LMArena Agent Arena SteerabilityAgents9.6-1.4−5.5
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine16.021.3+5.3
PubMedQA (SecureBio)Health & Medicine75.981.2+5.3
GDPval Extended (Mercor held-out rubric score)Professional Work58.964.1+5.2
FrontierCode 1.1 Extended - ScoreCode59.664.5+4.9
WitnessBench v1.0Games22.827.6+4.8
Wiz Cyber Model Arena v3.9Cybersecurity65.470.2+4.8
MedCodeHealth & Medicine53.248.5−4.7
APEX-SWECode43.948.6+4.7
Finance Agent v2 - PrecedentsFinance & Accounting25.630.3+4.7
ProgramBenchCode1.05.5+4.5
LisanBench Difficulty-WeightedGames26943113+4.5
Legal Research BenchLegal43.839.4−4.3
ACCoRDGeneral QA77.481.7+4.3
VariantBench V1 Clean RerunsScience42.446.6+4.2
WCB (SecureBio)Science66.762.6−4.1
LAB-Bench CloningScenarios (SecureBio)Science95.9100.0+4.1
MASKSafety94.498.5+4.1
Finance Agent v2 - Disclosure AnalysisFinance & Accounting65.761.7−4.0
BabaIsBench Stage 0: The IntroGames96.0100.0+4.0
BioSecBench-Function V0Science44.648.6+4.0
Humanity's Last ExamGeneral QA49.853.6+3.8
SpatialBench V0 VerifiedScience66.470.1+3.8
LLM Debate BenchmarkChat & Writing16311606−3.6
SWE Atlas - Codebase QnACode44.447.8+3.4
Finance Agent v2 - All PassFinance & Accounting43.540.3−3.3
SM BenchSafety72.969.9−3.0
SnakeBench Best ApplesGames28.027.0−2.9
CharXiv (descriptive and reasoning combined)Multimodal93.296.1+2.9
DistressBenchSafety71.474.3+2.9
SciCode (main problems)Science16.919.6+2.7
Finance Agent v2 - Earnings AnalysisFinance & Accounting72.169.5−2.6
GDPval-AA v2Professional Work15291580+2.5
EpiBench V1 Clean RerunsScience25.828.3+2.5
Ruby LLM Benchmarks — Program FixerCode72.675.1+2.5
DTBenchGeneral QA94.997.3+2.4
Excel Modeling BenchmarkFinance & Accounting69.471.7+2.3
GDPval (Mercor 213-task rubric score)Professional Work84.586.7+2.2
MedScribeHealth & Medicine85.887.9+2.2
SnakeBench Total ApplesGames226198−2.1
SciCode 1.0.1 (Artificial Analysis)Code54.456.5+2.1
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting66.468.4+2.0
HealthBenchHealth & Medicine58.856.9−1.9
Finance Agent v2 - Financial ModelingFinance & Accounting23.321.6−1.8
SpatialBench V1 Clean RerunsScience65.867.5+1.7
AIME 2025Math98.3100.0+1.7
Finance Agent v2 - Market AnalysisFinance & Accounting67.265.5−1.7
OTIS Mock AIME 2024-2025Math98.3100.0+1.7
TxBench Preclinical Pharmacology V1 Clean RerunsScience57.759.3+1.7
BioSecBench-Surveillance V1 Clean RerunsScience46.948.5+1.6
Tax Agent BenchFinance & Accounting64.963.3−1.6
MMLU ProGeneral QA90.091.5+1.5
WMDP-Bio (SecureBio)Science92.390.8−1.5
MetagenomicsBench V0Science48.349.7+1.4
LMArena Text Arena EloChat & Writing14821474−1.3
MetagenomicsBench V1Science48.349.7+1.3
ForecastBench Preliminary Dataset Brier IndexForecasting61.662.9+1.3
SEA-HELMGeneral QA75.676.9+1.3
SWE-TogetherCode61.560.6−0.9
LMArena Agent Arena Bash RecoveryAgents7.25.5−0.8
ForecastBench Tournament Overall Brier IndexForecasting66.266.9+0.7
ObviousBenchGeneral QA99.3100.0+0.7
Vision Arena OverallMultimodal12861283−0.7
Bio Capabilities Index (SecureBio)Indexes164164+0.6
Gray Swan IPI Attack Success (k=15)Safety8.08.5−0.5
Toloka Arena (v1)Agents54.353.8−0.5
MMLU-Bio (SecureBio)Science90.991.3+0.4
Harvey HLAB — Criteria Pass RateLegal87.987.5−0.4
Bench to the Future 3 — September Panel — PooledForecasting0.10.1−0.3
Finance Agent v2 - AdjustmentsFinance & Accounting48.448.3−0.1
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.00.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
Into the UnknownScience0.00.00.0

Opus 4.8vsGPT-6 Astranormalised score marginsData Nov 15, 2026Data: Vals AI, benchmarks.bio, Mercor and 70 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…