Atlas

Compare Models

GPT-6 Astra vs Grok 4.7

GPT-6 Astra leads by 6.6 capability points (≈95% interval +4.2 to +9.0): a clear lead over Grok 4.7 · P(higher) ≈ 100%

Wins 202 of 281 shared benchmarks (59 lost, 20 tied)

Capability vs. relative cost
GPT-6 AstraGrok 4.7Closed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

GPT-6 AstravsGrok 4.7capability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins202 – 59 · 281 shared, 20 tied

Comparing matching setups only (152 benchmarks excluded) · suites counted once
← GPT-6 AstraGrok 4.7 →

GPT-6 AstravsGrok 4.7wins by categoryData Nov 15, 2026Data: Vals AI, Artificial Analysis, Mercor and 69 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Grok 4.7 minus GPT-6 Astra.

Clipping Bench v1.0Games9514384−98.2
ARC-AGI-3 (Semi-Private)General QA99.910.0−89.9
FrontierMath Tier 4 (v2) - PrivateMath97.617.1−80.5
CyberGym-E2E-AA (Artificial Analysis)Cybersecurity0.074.0+74.0
ProofBench v1.1Math99.026.0−73.0
Concept Synth INDUCTION Challenge100 Round 1General QA94.024.0−70.0
Concept Synth INDUCTION Challenge64 Round 1General QA98.434.4−64.1
Furniture Assembly BenchmarkMultimodal80.020.8−59.2
ViBench 2.0Code60.31.5−58.8
ChartographyMultimodal71.014.7−56.3
Mystery Game PuzzlesGames84.029.0−55.0
Terminal-Bench Science 0.1 (Vals)Science65.711.4−54.3
WARC-Bench (192-task test split)Agents89.135.6−53.5
SRE Bench — Fully SolvedCode56.93.8−53.1
BrokenArXiv 05/2026Math91.040.0−51.0
SheetBench-50Finance & Accounting64.714.3−50.4
Terminal-Bench-Science 0.1Science59.59.5−50.0
ALE-BenchCode31641823−47.5
BrokenArXiv 06/2026Math99.152.8−46.3
Olam Social Poker EloGames15501508−45.7
Agent Initiative BenchmarkAgents9.04.5−44.6
AA-PersonalityBench Imagination indexChat & Writing77.235.3−42.3
IOI (Vals v2)Code100.057.7−42.3
FrontierMath Tiers 1-3 (v2) - PrivateMath93.753.0−40.7
ChessBench EloGames17551076−40.4
AA-PersonalityBench Organization indexChat & Writing69.529.6−40.4
Humanity's Last Exam DiamondGeneral QA63.823.6−40.2
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.040.0+40.0
FrontierSWE V2 Worst@5Code55.115.9−39.2
AA-PersonalityBench Conviction indexChat & Writing44.882.9+38.5
LAB-Bench ProtocolQA (SecureBio)Science89.351.4−37.9
LAB-Bench LitQA2 (SecureBio)Science54.816.9−37.9
OpenWorldEngineer (Mean@8)Code44.27.0−37.2
HardGeoBench Avg DistanceMultimodal9082104−36.9
AA-PersonalityBench Transparency indexChat & Writing94.358.1−36.5
TaxCalcBench TY25Finance & Accounting44.08.0−36.0
Toloka Arena (v3)Agents67.031.0−36.0
HoneyBench v0.2 (hack rate)Safety50.885.1−34.4
ArXivMath 05/2026Math95.060.8−34.2
LLM Creative Story WritingChat & Writing3.40.4−34.0
Terminal-Bench 4.0 (Artificial Analysis)Code59.625.8−33.8
WeirdML v3Science42.28.5−33.8
ARC-AGI-2General QA95.061.4−33.6
CheatBenchSafety45.078.0−33.0
Chess Puzzles (task version 2.0)Games74.041.0−33.0
LLM Chess EloGames1614822−32.5
FrontierSWE V2 Best@5Code73.041.1−31.9
VIALSMultimodal49.918.2−31.7
AA-PersonalityBench Warmth indexChat & Writing75.344.8−30.8
Vals Science IndexIndexes65.134.8−30.3
CyberBench v1.1 OverallCybersecurity41.169.5+28.4
MysteryMechanismScience53.225.2−27.9
BuseyBench SVGMultimodal7.34.6−27.5
RSI Arena Stage 1 human ratingAgents10951038−27.4
Code Migration v1.1Code80.653.2−27.4
DeepsecBench — Revalidated V1Code37.810.9−26.9
DeepsecBench — Revalidated V1 (233 issues)Code37.610.9−26.8
DataCenterBench Tier 1Agents40.013.3−26.7
GeneBench-Pro Public Release SubsetScience36.010.0−26.0
EnterpriseBench: CoreCraftProfessional Work42.668.2+25.6
AA-3D v1.0 EloCode1168992−25.5
Code2Video-AA EloCode14271127−24.4
ArXivMath 06/2026Math94.470.1−24.3
RiemannBench (November 2026 rescoring)Math81.657.6−24.0
HardGeoBench <=100miMultimodal51.327.5−23.7
Artificial Analysis Cyber Index v1Indexes33.456.4+23.0
GBENCH One-shot Coding GScoreGames79.256.3−23.0
Code MigrationCode67.744.8−22.9
LMArena Agent Arena Net Improvement (Steering Burden)Agents14.52.6−22.6
Short Form Bench v1.0Games28032578−22.3
FrontierCUA pass@1Agents42.220.0−22.2
GameBench 2Games94.372.3−22.0
DistressBenchSafety74.352.4−21.9
Artificial Analysis Intelligence Index v5.0.1Indexes56.935.5−21.4
ABC-Bench liquid handling (SecureBio)Science76.798.0+21.3
Extended NYT Connections 940 (Quadratic v1)Games98.176.8−21.3
LMArena Agent Arena Net ImprovementAgents10.44.0−20.9
Terminal-Bench 4.0Code57.937.6−20.3
Artificial Analysis Engineering IndexIndexes61.841.8−20.0
HardGeoBench <=10miMultimodal37.517.5−20.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal20.00.0−20.0
Medical Long Context Reasoning (MLCR-AA)Health & Medicine35.015.0−20.0
Artificial Analysis Intelligence Index v5.0Indexes55.335.3−19.9
LAB-Bench CloningScenarios (SecureBio)Science100.080.1−19.9
Roboflow Vision Evals (October 2026, five tasks)Multimodal83.463.5−19.9
SimpleQA Verified (Epoch anti-abstention)General QA75.656.0−19.6
sudo L7 (October 2026 rescoring)Code31.112.2−18.9
BrowseComp (Mercor 130-question subset)Search94.275.8−18.4
MBCT (SecureBio)Science63.946.1−17.8
MedXpertQA MM (200-question set)Health & Medicine86.869.0−17.8
AA-PersonalityBench Expressiveness indexChat & Writing58.175.3+17.3
Blueprint-Bench 2Multimodal0.50.3−17.2
Bio Capabilities Index (SecureBio)Indexes164141−16.9
Hermes IndexIndexes56.339.3−16.9
CritPt v2 (agentic, avg@5)Science78.962.3−16.6
SM BenchSafety69.986.3+16.4
PLE (Physical-Engineering's Last Exam)Professional Work48.331.9−16.4
HardGeoBench Country AccuracyMultimodal60.043.8−16.2
MentalHealthBenchSafety57.341.3−16.0
WellnessBenchSafety64.048.0−16.0
VCT v2 (SecureBio)Science58.042.0−16.0
Snorkel Agentic Coding 2.0Code47.631.9−15.7
Physical-Engineering's Last Exam (hidden test)Professional Work45.830.2−15.6
HPCT (SecureBio)Science69.053.7−15.3
LMCAGeneral QA64.449.4−15.0
Conceptual Reasoning IndexIndexes73.963.1−14.8
Roboflow Vision Evals (September 2026 protocol)Multimodal86.671.9−14.7
DAYJOB: Data ScienceProfessional Work21.36.7−14.6
Toloka Arena (v1)Agents53.839.3−14.5
LMArena Agent Arena Praise vs ComplaintAgents31.73.1−14.3
τ³-Banking (Mercor run, bm25_grep retrieval)Agents46.732.6−14.1
CloudOpsBench v1.0Agents42.856.5+13.7
CritPt (avg@5)Science31.718.0−13.7
RedlineBenchLegal62.849.5−13.3
FrontierCode 2.0 (Artificial Analysis)Code41.428.7−12.7
Ivo-micro1 ContractBenchLegal45.658.3+12.7
HANDBOOK.md Agents v1.1Professional Work31.444.0+12.6
LiveBench 2.0Agents86.174.1−12.0
CadQueryEval (Corrected Geometry)Code100.088.0−12.0
SnorkelLegalLegal20.932.8+11.9
ChessBench AccuracyGames85.074.0−11.8
Artificial Analysis Economics IndexIndexes57.245.7−11.5
HLAB-Vals-1.1Legal7.118.3+11.2
APEX-AccountingFinance & Accounting57.946.7−11.2
ChessBench CoherenceGames97.086.3−10.7
Epoch Capabilities Index (ECI)Indexes169156−10.6
Harvey LAB (Mercor run, undisclosed ~213-task cohort)Legal9.219.5+10.3
Design Creativity BenchCode65.562.2−10.3
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting78.868.6−10.2
MKA (Artificial Analysis)Professional Work50.240.1−10.1
DeepsecBench-AA (Artificial Analysis)Cybersecurity36.926.9−10.0
BioMysteryBench (Vals Terminus 2)Science79.369.3−10.0
DAYJOB: AccountingFinance & Accounting30.020.0−10.0
Humanity's Last Exam (Text-Only)General QA53.143.1−9.9
Artificial Analysis Progress Index v1Indexes350323−9.4
Senior SWE-BenchCode36.827.4−9.4
SnorkelUnderwrite 2.0Finance & Accounting21.230.5+9.3
DAYJOB: LawLegal36.927.6−9.3
GDP.pdf (Artificial Analysis, All-pass)Professional Work32.223.2−9.0
Eval Connections One-Shot with TrapsGames94.085.0−9.0
Vision Arena OverallMultimodal12831249−8.8
ComplexConstraintsChat & Writing57.748.9−8.8
MedXpertQA MM Extended (Mercor held-out)Health & Medicine60.852.0−8.8
SnorkelFinance 2.0Finance & Accounting11.620.2+8.6
LMArena Agent Arena Steering BurdenAgents14.8-2.3−8.5
Artificial Analysis Legal IndexIndexes55.647.2−8.4
ARC-AGI-1 Semi-PrivateGeneral QA98.590.2−8.3
Creative Writing v3 EloChat & Writing21772012−8.3
MMMU ProMultimodal86.478.2−8.3
AutoResearchExam (hidden-test AUARC)Agents0.60.5−7.7
AA-PersonalityBench Slop indexChat & Writing16.28.6−7.7
Legal Research BenchLegal39.447.1+7.7
ACCoRDGeneral QA81.774.0−7.6
SWE Atlas v1.1 - OverallCode69.462.1−7.3
Westworld Finance Diligence BenchFinance & Accounting54.547.2−7.3
Ruby LLM Benchmarks — Program FixerCode75.168.0−7.1
Artificial Analysis Healthcare & Medical IndexIndexes53.746.6−7.1
DAYJOB: FinanceFinance & Accounting21.514.5−7.0
CharXiv Extended (Mercor held-out)Multimodal85.178.2−6.9
DualEntry Accounting AI BenchmarkFinance & Accounting81.288.1+6.9
Financial Coaching BenchFinance & Accounting67.574.1+6.5
ITBench-AAAgents48.642.1−6.5
APEX-Agents 1.1Professional Work75.168.7−6.4
Vals Index v2Indexes66.660.2−6.4
GDPval-AA v2.1Professional Work15721717+6.4
ATLAS-Finance v1.0Finance & Accounting11.35.0−6.3
Snorkel SWE-bench CLICode12.46.1−6.3
o11y-bench 1.1 Pass^3Agents82.876.6−6.2
Artificial Analysis Intelligence Index v4.3.2Indexes52.746.4−6.2
AA-Omniscience IndexIndexes43.732.0−5.9
GDPxProfessional Work9.63.9−5.7
Artificial Analysis Finance & Accounting IndexIndexes57.451.7−5.7
Tax Agent BenchFinance & Accounting63.369.0+5.7
LMArena Text Arena EloChat & Writing14741442−5.6
SAGEMath46.440.8−5.6
FrontierCode 1.1 Extended - ScoreCode64.559.4−5.1
Finance Agent v2 - Earnings AnalysisFinance & Accounting69.574.6+5.1
AA-Briefcase v1.1 EloAgents15431636+5.1
Arena Alignment IndexSafety87.882.7−5.1
CWE-Bench-AA (Artificial Analysis)Cybersecurity63.368.3+5.0
ProgramBenchCode5.50.5−5.0
WCB (SecureBio)Science62.657.6−5.0
GPQA-Bio (SecureBio)Science87.482.5−4.9
Terminal-Bench 2.1Code89.985.0−4.9
APEX-SWECode48.653.4+4.8
Excel Modeling BenchmarkFinance & Accounting71.767.0−4.7
Ramp Accounting BenchFinance & Accounting48.744.1−4.6
SWE-TogetherCode60.665.1+4.5
LMArena Agent Arena Confirmed SuccessAgents14.05.1−4.5
Finance Agent v2 - ComparablesFinance & Accounting37.842.2+4.5
Visa Agentic Commerce BenchmarkAgents64.460.0−4.4
Finance Agent v2 - PrecedentsFinance & Accounting30.325.9−4.3
Vals Legal IndexIndexes68.472.7+4.3
DAXBench v2Code96.892.6−4.3
CharXiv (descriptive and reasoning combined)Multimodal96.192.0−4.1
Finance Agent v2 - Financial ModelingFinance & Accounting21.617.7−3.9
APEX-1 (Mercor, mean score)Professional Work69.365.4−3.9
AA-PersonalityBench Informativeness indexChat & Writing29.633.4+3.8
AA-AnalystAgent (pass^5)Professional Work51.355.0+3.8
Snorkel Digital Work IndexIndexes23.019.3−3.7
Finance Agent v2 - Market AnalysisFinance & Accounting65.561.8−3.7
BioSimBench 1.0Science15.912.3−3.6
DAYJOB: HealthcareHealth & Medicine11.68.4−3.2
Vals Finance IndexIndexes55.658.8+3.2
Artificial Analysis Strategy & Ops IndexIndexes58.054.8−3.1
DeepSWE v1.1Code74.171.0−3.1
GPQA DiamondGeneral QA95.892.7−3.0
Harvey HLAB — Criteria Pass RateLegal87.590.5+3.0
FrontierMath ErdősMath2.90.0−2.9
AutomationBench-AA 1.0.6Agents68.565.6−2.9
Finance Agent v2 - Disclosure AnalysisFinance & Accounting61.764.5+2.8
HealthBenchHealth & Medicine56.954.2−2.7
MMLU ProGeneral QA91.588.8−2.7
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine21.318.7−2.7
Dual-Arm Sim (task completion)Agents2.50.0−2.5
MissionReportProfessional Work61.859.5−2.3
HealthBench (length-adjusted, Artificial Analysis)Health & Medicine58.256.0−2.2
ForecastBench Preliminary Dataset Brier IndexForecasting62.960.7−2.2
SnorkelManufacturingProfessional Work10.012.1+2.1
Finance Agent v2 - AdjustmentsFinance & Accounting48.346.3−2.1
LLM Debate BenchmarkChat & Writing16061620+2.0
PubMedQA (SecureBio)Health & Medicine81.279.2−2.0
WMDP-Bio (SecureBio)Science90.888.8−2.0
OTIS Mock AIME 2024-2025Math100.098.1−1.9
Finance Agent v2 - All PassFinance & Accounting40.342.1+1.9
AdvancedIFChat & Writing82.080.2−1.8
MedScribeHealth & Medicine87.989.4+1.5
MetagenomicsBench V0Science49.748.3−1.4
DTBenchGeneral QA97.396.0−1.3
Political Science LLM BenchmarkProfessional Work67.966.6−1.3
SciCode 1.0.1 (Artificial Analysis)Code56.557.8+1.3
Excel Modeling Benchmark v1.1Finance & Accounting72.771.5−1.2
DeepResearch Bench II (Mercor 130-task set)Search50.149.0−1.1
MedCodeHealth & Medicine48.549.6+1.1
LMArena Agent Arena SteerabilityAgents-1.40.5+0.9
LegalReasonBenchLegal39.540.4+0.9
LMArena Agent Arena Bash RecoveryAgents5.57.2+0.8
Snorkel WorkplaceAgentsProfessional Work14.415.2+0.8
SWE Atlas - Codebase QnACode47.847.0−0.8
LMArena Agent Arena Bash ReliabilityAgents7.89.3+0.7
Hermes BenchAgents73.774.4+0.7
ObviousBenchGeneral QA100.099.3−0.7
GDP.xlsxProfessional Work22.922.3−0.6
SWE-bench Multimodal (Vals v1, 480 Tasks)Code43.342.9−0.4
SnorkelRevOpsProfessional Work10.911.3+0.4
MMLU-Bio (SecureBio)Science91.390.9−0.4
Immigration BenchLegal67.768.0+0.4
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting68.468.7+0.3
Longform Creative WritingChat & Writing82.883.0+0.2
DAYJOB: TaxFinance & Accounting4.85.0+0.2
Finance Agent v2.1Finance & Accounting60.460.6+0.2
SpeciEval (Refusal-Aware Retries, Historical)Safety96.596.4−0.1
SpeciEval (Pooled English Runs)Safety96.596.4−0.1
Creative Writing v3 Rubric ScoreChat & Writing15.915.9+0.1
LMArena Agent Arena Tool HallucinationAgents1.01.00.0
AA-LCR v1.1Search80.080.00.0
AIME 2025Math100.0100.00.0
BioResearchBenchScience4.04.00.0
CVDP (517-problem set)Code54.954.90.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal20.020.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.00.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0

GPT-6 AstravsGrok 4.7normalised score marginsData Nov 15, 2026Data: Vals AI, Artificial Analysis, Mercor and 69 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…