Atlas

Compare Models

Gemini 3.7 Flash vs GPT-6 Astra

GPT-6 Astra leads by 7.9 capability points (≈95% interval +5.5 to +10.2): a clear lead over Gemini 3.7 Flash · P(higher) ≈ 100%

Wins 176 of 236 shared benchmarks (42 lost, 18 tied)

Capability vs. relative cost
Gemini 3.7 FlashGPT-6 AstraClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Gemini 3.7 FlashvsGPT-6 Astracapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins42 – 176 · 236 shared, 18 tied

Comparing matching setups only (57 benchmarks excluded) · suites counted once
← Gemini 3.7 FlashGPT-6 Astra →

Gemini 3.7 FlashvsGPT-6 Astrawins by categoryData Nov 15, 2026Data: Vals AI, benchmarks.bio, Artificial Analysis and 51 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6 Astra minus Gemini 3.7 Flash.

Olam Social Poker EloGames14681550+89.1
BrokenArXiv 06/2026Math11.199.1+88.0
Concept Synth INDUCTION Challenge100 Round 1General QA11.094.0+83.0
BrokenArXiv 05/2026Math8.591.0+82.5
Concept Synth INDUCTION Challenge64 Round 1General QA17.298.4+81.3
Odysseys2Agents5.884.6+78.8
BrokenArXiv 04/2026Math19.397.5+78.3
FrontierMath Tier 4 (v2) - PrivateMath36.697.6+61.0
Terminal-Bench Science 0.1 (Vals)Science5.765.7+60.0
Furniture Assembly BenchmarkMultimodal26.780.0+53.3
Terminal-Bench 4.0Code6.157.9+51.8
ArXivMath 05/2026Math47.595.0+47.5
Mystery Game PuzzlesGames37.084.0+47.0
LLM Creative Story WritingChat & Writing-0.73.4+46.9
ALE-BenchCode18763164+45.6
FrontierSWE V2 Worst@5Code10.355.1+44.8
MASKSafety53.798.5+44.8
FrontierSWE V2 Best@5Code31.273.0+41.8
ProofBench v1.1Math58.099.0+41.0
Terminal-Bench 4.0 (Artificial Analysis)Code13.654.0+40.4
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal40.00.0−40.0
LMArena Agent Arena Net ImprovementAgents-1.310.4+38.5
HPCT Refusal (CAIS)Safety13.050.0+37.0
EnigmaEvalGeneral QA33.969.7+35.8
ArXivMath 04/2026Math57.590.8+33.3
Code MigrationCode34.867.7+32.9
RiemannBenchMath39.272.0+32.8
Humanity's Last Exam DiamondGeneral QA31.263.8+32.6
IOI (Vals v2)Code67.8100.0+32.2
ArXivMath 06/2026Math62.594.4+31.9
LMArena Agent Arena Net Improvement (Steering Burden)Agents-2.214.5+31.6
ChartographyMultimodal40.471.0+30.6
TaxCalcBench TY25Finance & Accounting14.044.0+30.0
AutoformBenchMath52.582.4+29.9
ChessBench EloGames12651755+29.1
Machiavelli Deception (CAIS)Safety85.576.3+27.8
GlobeBenchGeneral QA21.849.6+27.8
GBENCH One-shot Coding GScoreGames52.679.2+26.6
Agent Red Teaming k=100 (CAIS)Safety38.513.5+25.0
MMMU-Pro Extended (Mercor held-out)Multimodal36.460.0+23.6
VIALSMultimodal26.549.9+23.4
Creative Writing v3 EloChat & Writing17202177+23.1
FrontierMath Tiers 1-3 (v2) - PrivateMath71.693.7+22.1
Senior SWE-BenchCode14.736.8+22.1
GeneBench-Pro Public Release SubsetScience14.036.0+22.0
AA-Briefcase EloAgents11161494+22.0
LLM Debate BenchmarkChat & Writing14551606+21.4
TextQuests Game ProgressGames52.974.2+21.3
LLM Chess EloGames11071614+20.8
Harvey's Legal Agent Benchmark - Banking FinanceLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - InsuranceLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.020.0+20.0
Medical Long Context Reasoning (MLCR-AA)Health & Medicine15.035.0+20.0
LiveBench 2.0Agents67.286.1+19.0
DAYJOB: FinanceFinance & Accounting2.821.5+18.7
ScBench-Long V0 VerifiedScience24.242.4+18.2
NoCoT-Bench NCRI 15.2General QA126159+18.1
MetagenomicsBench V0Science31.749.7+18.0
MetagenomicsBench V1Science31.749.7+18.0
HardGeoBench Avg DistanceMultimodal8951478−18.0
Learn2Play BenchGames58.376.2+17.9
SpatialViz-BenchMultimodal75.393.1+17.8
RedlineBenchLegal45.362.8+17.5
CritPt (avg@5)Science14.331.7+17.4
LMArena Agent Arena Praise vs ComplaintAgents-2.931.7+17.3
TextQuests Average Cumulative HarmSafety24.429.2−16.3
CadQueryEval (Corrected Geometry)Code84.0100.0+16.0
TxBench Oligo Discovery V0Science39.555.5+15.9
Remote Labor Index (RLI)Professional Work5.020.8+15.8
ComplexConstraintsChat & Writing42.457.7+15.3
VariantBench V1 Clean RerunsScience31.446.6+15.3
HackerRank-BenchCode28.543.5+15.0
HieroglyphBenchMultimodal55.040.0−15.0
BrowseComp (Mercor 130-question subset)Search79.294.2+15.0
HardGeoBench Country AccuracyMultimodal67.552.5−15.0
BioSecBench-Function V1 Clean RerunsScience33.748.6+15.0
Eval Connections One-Shot with TrapsGames80.094.0+14.0
BioSecBench-Refusal V1 Clean RerunsSafety54.840.8−14.0
LMCAGeneral QA50.464.4+14.0
Humanity's Sixth SenseMultimodal39.853.6+13.8
ChessBench CoherenceGames83.297.0+13.8
Artificial Analysis Intelligence Index v4.3.2Indexes39.152.7+13.6
TxBench Oligo Discovery V1 Clean RerunsScience35.649.2+13.6
ScBench V1 Clean RerunsScience50.463.9+13.5
Artificial Analysis Intelligence Index v4.3Indexes39.452.8+13.4
Arena Alignment IndexSafety74.487.8+13.4
EnterpriseBench: CoreCraftProfessional Work55.942.6−13.3
Conceptual Reasoning IndexIndexes64.673.9+12.8
VCT-Refusal (CAIS)Safety4.817.6+12.8
Finance Agent v2 - ComparablesFinance & Accounting50.337.8−12.5
SpatialBench-Long V0 VerifiedScience22.734.9+12.1
DAYJOB: HealthcareHealth & Medicine0.011.6+11.6
Artificial Analysis Progress Index v1Indexes317350+11.3
Wiz Cyber Model Arena v3.9Cybersecurity59.170.2+11.1
LMArena Agent Arena Steering BurdenAgents-7.314.8+11.1
SM BenchSafety80.669.9−10.7
ARC-AGI-2General QA84.695.0+10.4
HardGeoBench <=100miMultimodal56.346.3−10.0
HardGeoBench <=10miMultimodal46.336.3−10.0
Finance Agent v2 - Earnings AnalysisFinance & Accounting79.169.5−9.6
SpatialBench-Long V1 Clean RerunsScience29.839.3+9.5
VCT v2 (SecureBio)Science48.658.0+9.4
IntPhys 2Multimodal68.277.4+9.2
VCT (SecureBio)Science49.358.5+9.2
Toloka Arena (v3)Agents58.067.0+9.0
Finance Agent v2 - Market AnalysisFinance & Accounting74.565.5−9.0
MBCT (SecureBio)Science55.063.9+8.9
LMArena Agent Arena Confirmed SuccessAgents-3.514.0+8.8
AA-AnalystAgent (pass^5)Professional Work60.051.3−8.8
Epoch Capabilities Index (ECI)Indexes158169+8.7
AA-Omniscience IndexIndexes26.543.7+8.6
GDP.pdf (Artificial Analysis, All-pass)Professional Work23.632.2+8.6
ScBench-Long V1 Clean RerunsScience18.827.3+8.5
BioSecBench-Function V0Science40.148.6+8.5
SpatialBench V1 Clean RerunsScience59.167.5+8.4
AHK-EvalCode91.7100.0+8.3
Document ArenaProfessional Work14761467−8.2
FrontierCode 1.1 Extended - ScoreCode56.364.5+8.2
Chess Puzzles (task version 2.0)Games66.074.0+8.0
Humanity's Last ExamGeneral QA45.653.6+8.0
PCT Political Inconsistency (CAIS)Safety45.137.4+7.7
𝜏³-BankingAgents35.543.1+7.6
APEX-Accounting 1.1Finance & Accounting57.264.7+7.5
SpeciEval (Pooled English Runs)Safety89.096.5+7.5
SpeciEval (Refusal-Aware Retries, Historical)Safety89.096.5+7.5
SUPERChemScience72.880.3+7.5
Vals Index v2Indexes59.366.6+7.3
DeepSWE v1.1Code65.572.8+7.3
Finance Agent v2 - All PassFinance & Accounting47.540.3−7.2
BioSecBench-Surveillance V1 Clean RerunsScience41.548.5+7.1
BioSecBench-Surveillance V0 (102 tasks)Science41.548.5+7.0
Shroombench (mushroom identification, top-1)Multimodal61.054.0−7.0
NoCoT-Bench NCKI (kspine v3)General QA121131+6.9
TxBench Antibody Discovery V0Science50.056.9+6.9
BuseyBench SVGMultimodal6.67.3+6.8
DeepResearch Bench II (Mercor 130-task set)Search43.450.1+6.7
BabaIsBench Stage 1: The LakeGames93.3100.0+6.7
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine14.721.3+6.7
APEX-SWECode42.048.6+6.6
AutomationBench-AA 1.0.6Agents62.068.5+6.5
Finance Agent v2 - AdjustmentsFinance & Accounting54.748.3−6.4
SimpleQA Verified (Epoch anti-abstention)General QA69.275.6+6.4
Ramp Accounting BenchFinance & Accounting42.348.7+6.4
ATLAS-Finance v1.1Finance & Accounting3.39.7+6.3
GDPval Extended (Mercor held-out rubric score)Professional Work57.864.1+6.3
EpiBench V1 Clean RerunsScience22.028.3+6.3
ChessBench AccuracyGames79.285.0+6.2
Finance Agent v2 - Disclosure AnalysisFinance & Accounting67.761.7−6.1
CharXiv Extended (Mercor held-out)Multimodal79.185.1+6.0
GDPval-AA v2.1Professional Work14041538+5.9
GPQA-Bio (SecureBio)Science81.687.4+5.8
SpatialBench V0 VerifiedScience64.370.1+5.8
Tax Agent BenchFinance & Accounting57.763.3+5.7
MirrorCode (ML, +Private, 2L)Code41.146.7+5.6
ProgramBenchCode0.05.5+5.5
LMArena Agent Arena Bash RecoveryAgents-5.45.5+5.5
DistressBenchSafety69.074.3+5.4
TxBench Antibody Discovery V1 Clean RerunsScience43.548.9+5.4
MedCodeHealth & Medicine53.448.5−4.9
Legal Research BenchLegal34.639.4+4.8
GDPval-AA v2Professional Work14331530+4.7
ACCoRDGeneral QA77.081.7+4.7
ChessBench — Historical Output RepairGames70.766.0−4.7
WCB (SecureBio)Science58.062.6+4.6
ERQAMultimodal77.782.3+4.6
BioTIER RefusalSafety84.479.9−4.5
FACTS Benchmark SuiteGeneral QA71.275.7+4.4
APEX-Agents 1.1Professional Work79.475.1−4.3
HPCT (SecureBio)Science64.769.0+4.3
MindCube TinyMultimodal88.492.7+4.3
Terminal-Bench 2.1Code85.889.9+4.1
TxBench Preclinical Pharmacology V1 Clean RerunsScience55.359.3+4.0
MedScribeHealth & Medicine83.987.9+4.0
Creative Writing v3 Rubric ScoreChat & Writing15.215.9+3.6
Finance Agent v2 - PrecedentsFinance & Accounting33.830.3−3.5
SciCode 1.0.1 (Artificial Analysis)Code59.856.5−3.4
LABBench2Science82.185.4+3.3
Roboflow Vision Evals (October 2026, five tasks)Multimodal80.183.4+3.3
AIME 2025Math96.7100.0+3.3
MedXpertQA MM (200-question set)Health & Medicine83.586.8+3.3
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting65.368.4+3.1
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting81.878.8−3.0
AA-LCR v1.1Search83.080.0−3.0
ARC-AGI-1 Semi-PrivateGeneral QA95.598.5+3.0
SAGEMath49.246.4−2.9
OTIS Mock AIME 2024-2025Math97.2100.0+2.8
CyberBench v1.1 OverallCybersecurity43.841.1−2.7
LMArena Text Arena EloChat & Writing14891474−2.7
Vision Arena OverallMultimodal12941283−2.7
Finance Agent v2 - Financial ModelingFinance & Accounting24.221.6−2.6
LAB-Bench ProtocolQA (SecureBio)Science86.889.3+2.5
APEX-1 (Mercor, mean score)Professional Work67.069.3+2.3
LVBenchMultimodal85.487.5+2.1
MedXpertQA MM Extended (Mercor held-out)Health & Medicine62.760.8−1.9
GPQA Diamond (Artificial Analysis avg@5)General QA94.596.3+1.7
Bio Capabilities Index (SecureBio)Indexes162164+1.6
MMMU ProMultimodal84.986.4+1.6
GDPval (Mercor 213-task rubric score)Professional Work85.286.7+1.5
Roboflow Vision Evals (September 2026 protocol)Multimodal85.286.6+1.4
ForecastBench Preliminary Dataset Brier IndexForecasting61.762.9+1.2
Harvey HLAB — Criteria Pass RateLegal88.687.5−1.1
MMLU ProGeneral QA90.691.5+0.9
LMArena Agent Arena Bash ReliabilityAgents6.27.8+0.8
Toloka Arena (v1)Agents54.653.8−0.8
CharXiv (descriptive and reasoning combined)Multimodal95.396.1+0.8
ForecastBench Tournament Overall Brier IndexForecasting66.266.9+0.7
Gray Swan IPI Attack Success (k=15)Safety9.28.5+0.7
LMArena Agent Arena SteerabilityAgents-0.0-1.4−0.7
LAB-Bench CloningScenarios (SecureBio)Science99.4100.0+0.6
LAB-Bench LitQA2 (SecureBio)Science54.254.8+0.6
DTBenchGeneral QA96.897.3+0.5
Excel Modeling BenchmarkFinance & Accounting71.371.7+0.4
LMArena Agent Arena Tool HallucinationAgents0.71.0+0.2
MMLU-Bio (SecureBio)Science91.491.3−0.1
BabaIsBench Stage 0: The IntroGames100.0100.00.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.00.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.00.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
ObviousBenchGeneral QA100.0100.00.0

Gemini 3.7 FlashvsGPT-6 Astranormalised score marginsData Nov 15, 2026Data: Vals AI, benchmarks.bio, Artificial Analysis and 51 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…