Atlas

Compare Models

GPT-6 Astra vs Grok 4.6

GPT-6 Astra leads by 7.2 capability points (≈95% interval +4.9 to +9.6): a clear lead over Grok 4.6 · P(higher) ≈ 100%

Wins 209 of 279 shared benchmarks (49 lost, 21 tied)

Capability vs. relative cost
GPT-6 AstraGrok 4.6Closed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

GPT-6 AstravsGrok 4.6capability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins209 – 49 · 279 shared, 21 tied

Comparing matching setups only (113 benchmarks excluded) · suites counted once
← GPT-6 AstraGrok 4.6 →

GPT-6 AstravsGrok 4.6wins by categoryData Nov 15, 2026Data: Vals AI, Artificial Analysis, benchmarks.bio and 68 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Grok 4.6 minus GPT-6 Astra.

Clipping Bench v1.0Games9514805−93.7
RT-SAFE Collisions per EpisodeSafety22.459.5−93.0
Time Horizon Index: KSPAgents90.55.8−84.7
LLM Creative Story WritingChat & Writing3.4-2.9−71.5
Design Creativity BenchCode65.543.4−70.1
Concept Synth INDUCTION Challenge100 Round 1General QA94.025.0−69.0
FrontierMath Tier 4 (v2) - PrivateMath97.631.7−65.9
Concept Synth INDUCTION Challenge64 Round 1General QA98.435.9−62.5
AA-PersonalityBench Imagination indexChat & Writing77.220.0−57.7
ARC-AGI-3 (Semi-Private)General QA59.32.1−57.2
AA-PersonalityBench Organization indexChat & Writing69.514.3−55.8
Olam Social Poker EloGames15501499−55.4
Terminal-Bench-Science 0.1Science59.56.2−53.3
IOI (Vals v2)Code100.047.6−52.4
Terminal-Bench Science 0.1 (Vals)Science62.911.4−51.4
ChessBench EloGames1755891−51.4
ChartographyMultimodal71.020.5−50.5
Mystery Game PuzzlesGames84.034.0−50.0
AA-PersonalityBench Conviction indexChat & Writing44.892.4+48.1
LLM Chess EloGames1614443−48.1
ProofBench v1.1Math99.051.0−48.0
ALE-BenchCode31641822−47.5
AA-PersonalityBench Warmth indexChat & Writing75.331.5−44.2
Gray Swan IPI Attack Success (k=15)Safety8.551.8−43.3
Terminal-Bench 4.0 (Artificial Analysis)Code59.617.2−42.4
Humanity's Last Exam DiamondGeneral QA63.821.5−42.3
FrontierSWE V2 Worst@5Code55.112.9−42.2
Chess Puzzles (task version 2.0)Games74.033.0−41.0
VoxelBench Text RatingGames26252050−41.0
Furniture Assembly BenchmarkMultimodal80.040.0−40.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal0.040.0+40.0
NoCoT-Bench NCRI 15.2General QA15986.8−39.7
FrontierSWE V2 Best@5Code73.037.3−35.7
ZeroBench (Main questions, pass@5)Multimodal52.017.0−35.0
CheatBenchSafety45.079.6−34.6
MMMU-Pro Extended (Mercor held-out)Multimodal60.026.0−34.0
Terminal-Bench 4.0Code57.925.3−32.6
ZeroBench Main Mean pass@1 (five samples)Multimodal43.211.0−32.2
LMArena Agent Arena Net ImprovementAgents10.41.1−30.5
VIALSMultimodal49.920.5−29.4
RareDiseaseBench V1 (35 evaluations)Science33.34.8−28.6
SCI-AGENT-EXAMScience80.952.4−28.5
CadQueryEval (Corrected Geometry)Code100.072.0−28.0
ZeroBench Main Questions pass^5Multimodal35.07.0−28.0
ARC-AGI-2General QA95.067.1−27.9
LMArena Agent Arena Net Improvement (Steering Burden)Agents14.5-0.2−27.8
ChessBench CoherenceGames97.069.2−27.8
HardGeoBench Avg DistanceMultimodal9081807−27.7
FrontierMath Tiers 1-3 (v2) - PrivateMath93.766.0−27.7
Long-Horizon Terminal-Bench 2.0 (mean reward)Code69.242.0−27.2
BuseyBench SVGMultimodal7.34.6−27.1
Vals Science IndexIndexes65.138.1−27.0
ChessBenchGames70.043.3−26.7
CyberBench v1.1 OverallCybersecurity41.167.7+26.6
SimpleQA Verified (Epoch anti-abstention)General QA75.649.3−26.3
Document ArenaProfessional Work14671438−26.1
WeirdML (v2)Science93.367.3−26.0
LAB-Bench LitQA2 (SecureBio)Science54.828.9−25.9
Artificial Analysis Intelligence Index v5.0.1Indexes56.931.0−25.8
GameBench 2Games94.368.7−25.6
CrossSearch (pass^3)Search60.035.0−25.0
Wiz Cyber Model Arena v3.9Cybersecurity70.245.5−24.7
Roboflow Vision Evals (October 2026, five tasks)Multimodal83.459.1−24.3
GBENCH One-shot Coding GScoreGames79.255.4−23.9
Code MigrationCode67.744.6−23.2
AA-PersonalityBench Transparency indexChat & Writing94.371.4−23.1
EnterpriseBench: CoreCraftProfessional Work42.665.6+23.0
Medical Long Context Reasoning (MLCR-AA)Health & Medicine35.012.2−22.8
MysteryMechanismScience53.230.6−22.5
FrontierCode 2.0 (Artificial Analysis)Code41.419.8−21.6
DeepsecBench — Revalidated V1Code37.816.2−21.6
DeepsecBench — Revalidated V1 (233 issues)Code37.616.1−21.5
GlobeBenchGeneral QA49.628.1−21.4
ComplexConstraintsChat & Writing57.736.5−21.2
LAB-Bench ProtocolQA (SecureBio)Science89.368.1−21.2
HealthBench ProfessionalHealth & Medicine69.748.5−21.2
CritPt v2 (agentic, avg@5)Science83.462.3−21.1
GeneBench-Pro Public Release SubsetScience36.015.0−21.0
WellnessBenchSafety64.043.0−21.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal20.00.0−20.0
Bio Capabilities Index (SecureBio)Indexes164137−20.0
SnorkelLegalLegal20.940.7+19.8
HieroglyphBenchMultimodal40.020.3−19.7
BioSecBench-RefusalSafety44.664.2+19.6
AA-PersonalityBench Slop indexChat & Writing16.235.3+19.2
SM BenchSafety69.989.0+19.1
Co-Design BenchCode69.951.1−18.8
HardGeoBench <=100miMultimodal51.332.5−18.7
ChessBench AccuracyGames85.067.8−18.4
ScBench-Long V0 VerifiedScience42.424.2−18.2
ORCA-bench Hard RCA (Fireworks 324-task subset)Agents50.232.1−18.1
Extended NYT Connections 940 (Quadratic v1)Games98.180.0−18.1
Roboflow Vision Evals (September 2026 protocol)Multimodal86.668.7−17.9
Hedge-BenchFinance & Accounting44.226.4−17.7
HardGeoBench <=10miMultimodal37.520.0−17.5
SAGEMath46.428.9−17.5
LMArena Agent Arena Praise vs ComplaintAgents31.7-2.5−17.1
TissueMicroscopyBench V1 (49 evaluations)Science19.72.7−17.0
Harvey LAB held-out set (Fireworks run)Legal5.122.1+16.9
Blueprint-Bench 2Multimodal0.50.3−16.5
HardGeoBench Country AccuracyMultimodal60.043.8−16.2
LMCAGeneral QA64.448.5−15.9
HackerRank-BenchCode43.527.7−15.8
Conceptual Reasoning IndexIndexes73.962.5−15.7
AA-PersonalityBench Informativeness indexChat & Writing29.644.8+15.4
SpatialBench-Long V0 VerifiedScience34.919.7−15.2
Humanity's Last ExamGeneral QA54.839.7−15.1
MBABench 2.0Finance & Accounting15.81.0−14.8
SUPERChemScience80.365.7−14.6
GDP.pdf (Artificial Analysis, All-pass)Professional Work32.217.8−14.4
RedlineBenchLegal62.848.5−14.3
Snorkel Agentic Coding 2.0Code47.633.6−14.0
SnorkelFinance 2.0Finance & Accounting11.625.1+13.5
MBCT (SecureBio)Science63.950.4−13.5
NoCoT-Bench NCKI (kspine v3)General QA131112−13.4
BabaIsBench Stage 1: The LakeGames100.086.7−13.3
SpatialBench-Long V1 Clean RerunsScience39.326.2−13.1
Vibe Code Bench 1–100Code27.614.8−12.9
MedXpertQA MM Extended (Mercor held-out)Health & Medicine60.848.4−12.4
WCB (SecureBio)Science62.650.3−12.3
CritPt (avg@5)Science31.719.7−12.0
MedXpertQA MM (200-question set)Health & Medicine86.875.2−11.6
TxBench Oligo Discovery V0Science55.544.0−11.5
LiveBench 2.0Agents86.174.9−11.2
ARC-AGI-1 Semi-PrivateGeneral QA98.587.5−11.0
MetagenomicsBench V0Science49.738.7−11.0
MetagenomicsBench V1Science49.738.7−11.0
Toloka Arena (v3)Agents67.056.0−11.0
ScBench-Long V1 Clean RerunsScience27.316.5−10.8
AutoResearchExam (hidden-test AUARC)Agents0.60.5−10.7
DAXBench v2Code96.886.2−10.6
VariantBenchScience48.638.1−10.5
Senior SWE-BenchCode36.826.3−10.5
EpiBenchScience44.634.3−10.4
BrowseComp (Mercor 130-question subset)Search94.284.0−10.2
VCT (SecureBio)Science58.548.3−10.2
Bedside BenchHealth & Medicine93.483.3−10.1
ACCoRDGeneral QA81.771.6−10.1
Shroombench (mushroom identification, top-1)Multimodal54.044.0−10.0
AA-AnalystAgent (pass^5)Professional Work51.341.3−10.0
Artificial Analysis Progress Index v1Indexes350322−9.8
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting68.477.8+9.3
APEX-SWECode48.657.7+9.1
Epoch Capabilities Index (ECI)Indexes169158−9.1
Excel Modeling BenchmarkFinance & Accounting71.762.7−9.0
DAYJOB: HealthcareHealth & Medicine11.62.8−8.8
DAYJOB: FinanceFinance & Accounting21.512.8−8.7
Legal Research BenchLegal39.448.1+8.7
Artificial Analysis Intelligence Index v4.3Indexes52.844.4−8.4
Finance Agent v2 - PrecedentsFinance & Accounting30.321.9−8.4
Artificial Analysis Intelligence Index v4.3.2Indexes52.744.3−8.4
LMArena Agent Arena Confirmed SuccessAgents14.0-2.6−8.3
LMArena Agent Arena Steering BurdenAgents14.8-1.8−8.3
Tax Agent BenchFinance & Accounting63.371.4+8.0
CharXiv Extended (Mercor held-out)Multimodal85.177.3−7.8
SimpleBenchGeneral QA83.675.9−7.7
𝜏³-BankingAgents43.150.7+7.6
Vals Index v2Indexes66.659.2−7.4
HPCT (SecureBio)Science69.061.6−7.4
DeepSWE v1.1Code74.166.7−7.4
BioMysteryBench (Vals Terminus 2)Science79.372.2−7.0
Arena Alignment IndexSafety87.880.8−7.0
Snorkel SWE-bench CLICode12.45.7−6.7
TxBench-Antibody-Development V1 (60 evaluations)Science32.826.1−6.7
AA-Omniscience IndexIndexes43.730.5−6.6
BioSecBench-Function V0Science48.642.0−6.6
SnorkelManufacturingProfessional Work10.016.4+6.4
SciUniverse Level 1Science32.526.2−6.2
VariantBench V1 Clean RerunsScience46.640.4−6.2
BioSecBench-Refusal V1 Clean RerunsSafety40.846.9+6.1
MMMU ProMultimodal86.480.5−5.9
TxBench Oligo Discovery V1 Clean RerunsScience49.243.3−5.8
LAB-Bench CloningScenarios (SecureBio)Science100.094.2−5.8
AA-PersonalityBench Expressiveness indexChat & Writing58.163.8+5.8
BioSecBench-Structure V1 (58 evaluations)Science21.315.5−5.7
SpeciEval (Refusal-Aware Retries, Historical)Safety96.590.8−5.7
SpeciEval (Pooled English Runs)Safety96.590.8−5.7
MirrorCode (ML, +Private, 2L)Code46.741.1−5.6
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting78.873.2−5.5
LibraryDesignBenchCode45.139.7−5.4
GDPval-AA v2Professional Work15551663+5.3
Vision Arena OverallMultimodal12831263−5.2
TxBench Antibody Discovery V0Science56.951.9−5.0
Harvey HLAB — Criteria Pass RateLegal87.592.5+5.0
SciCode (main problems)Science19.614.6−5.0
ATLAS-Finance v1.1Finance & Accounting9.74.7−5.0
SnorkelRevOpsProfessional Work10.915.8+4.9
ScBench V1 Clean RerunsScience63.959.1−4.8
EpiBench V1 Clean RerunsScience28.323.6−4.7
Harvey LAB: In-House Contracting (held-out)Legal5.310.0+4.7
BioSecBench-Function V1 Clean RerunsScience48.644.0−4.6
ProgramBenchCode5.51.0−4.5
APEX-Accounting 1.1Finance & Accounting64.760.3−4.4
Finance Agent v2 - Earnings AnalysisFinance & Accounting69.573.9+4.4
TxBench Antibody Discovery V1 Clean RerunsScience48.944.5−4.4
Toloka Arena (v1)Agents53.849.7−4.1
BabaIsBench Stage 0: The IntroGames100.096.0−4.0
GDPval-AA v2.1Professional Work15721658+3.8
MedCodeHealth & Medicine48.544.7−3.8
micro1 PharmaBenchHealth & Medicine16.112.5−3.6
Finance Agent v2 - ComparablesFinance & Accounting37.841.2+3.5
SnorkelUnderwrite 2.0Finance & Accounting21.217.8−3.4
Finance Agent v2 - Market AnalysisFinance & Accounting65.562.2−3.3
CharXiv (descriptive and reasoning combined)Multimodal96.192.9−3.2
FrontierCode 1.1 Extended - ScoreCode64.561.3−3.2
CloudOpsBench v1.0Agents42.839.7−3.1
CVDP (517-problem set)Code54.958.0+3.1
LMArena Text Arena EloChat & Writing14741456−3.1
Eval Connections One-Shot with TrapsGames94.091.0−3.0
AHK-EvalCode100.097.2−2.8
Snorkel WorkplaceAgentsProfessional Work14.417.1+2.7
Finance Agent v2 - Financial ModelingFinance & Accounting21.618.9−2.7
Finance Agent v2 - Disclosure AnalysisFinance & Accounting61.764.4+2.7
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine21.318.7−2.7
Finance Agent v2 - All PassFinance & Accounting40.342.9+2.6
Snorkel Digital Work IndexIndexes23.020.5−2.5
SWE Atlas - Codebase QnACode47.850.3+2.5
TxBench Preclinical Pharmacology V1 Clean RerunsScience59.361.7+2.3
LMArena Agent Arena SteerabilityAgents-1.43.3+2.3
GDPval Extended (Mercor held-out rubric score)Professional Work64.161.8−2.3
APEX-Agents 1.1Professional Work75.177.3+2.2
NeuroscienceBench V1 (91 evaluations)Science22.720.5−2.2
MacroscopeBenchCode78.075.9−2.1
MMLU ProGeneral QA91.589.4−2.1
ForecastBench Preliminary Dataset Brier IndexForecasting62.961.0−1.9
BioSecBench-RespiratoryVirus-Surveillance V1 (110 evaluations)Science13.311.5−1.8
GPQA DiamondGeneral QA95.894.0−1.8
Immigration BenchLegal67.766.0−1.7
APEX-1 (Mercor, mean score)Professional Work69.367.7−1.6
Terminal-Bench 2.1Code89.988.4−1.5
AutomationBench-AA 1.0.6Agents68.567.0−1.5
MedScribeHealth & Medicine87.986.5−1.4
Finance Agent v2 - AdjustmentsFinance & Accounting48.349.6+1.3
ForecastBench Tournament Overall Brier IndexForecasting66.965.8−1.1
AdvancedIFChat & Writing82.080.9−1.1
LMArena Agent Arena Bash ReliabilityAgents7.810.0+1.1
AA-LCR v1.1Search80.081.0+1.0
BioSecBench-Surveillance V0 (102 tasks)Science48.547.5−1.0
BioSecBench-Surveillance V1 Clean RerunsScience48.547.5−1.0
OTIS Mock AIME 2024-2025Math100.099.2−0.8
GDPval (Mercor 213-task rubric score)Professional Work86.786.0−0.7
AA-Briefcase EloAgents15341545+0.7
SAB ResearchFinance & Accounting95.294.6−0.6
Prophet Arena Arena Score (Agentic, OpenCode)Forecasting86.285.6−0.6
MMLU-Bio (SecureBio)Science91.391.8+0.5
SpatialBench V1 Clean RerunsScience67.567.3−0.3
SpatialBench V0 VerifiedScience70.169.9−0.3
Prophet Arena Arena Score (Fixed Context)Forecasting84.785.0+0.3
LLM Debate BenchmarkChat & Writing16061604−0.2
LMArena Agent Arena Bash RecoveryAgents5.55.7+0.1
LMArena Agent Arena Tool HallucinationAgents1.01.00.0
AIME 2025Math100.0100.00.0
DTBenchGeneral QA97.397.30.0
GPQA-Bio (SecureBio)Science87.487.40.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal20.020.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
ObviousBenchGeneral QA100.0100.00.0
SciCode 1.0.1 (Artificial Analysis)Code56.556.50.0
SWE-TogetherCode60.660.60.0

GPT-6 AstravsGrok 4.6normalised score marginsData Nov 15, 2026Data: Vals AI, Artificial Analysis, benchmarks.bio and 68 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…