Atlas

Compare Models

Gemini 3.7 Flash vs Grok 4.7

Grok 4.7 leads by 1.2 capability points (≈95% interval −0.8 to +3.2): statistically tied · P(higher) ≈ 88%

Wins 92 of 190 shared benchmarks (83 lost, 15 tied)

Capability vs. relative cost
Gemini 3.7 FlashGrok 4.7Closed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Gemini 3.7 FlashvsGrok 4.7capability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins83 – 92 · 190 shared, 15 tied

Comparing matching setups only (25 benchmarks excluded) · suites counted once
← Gemini 3.7 FlashGrok 4.7 →

Gemini 3.7 FlashvsGrok 4.7wins by categoryData Nov 15, 2026Data: Vals AI, Arena, Mercor and 43 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Grok 4.7 minus Gemini 3.7 Flash.

Olam Social Poker EloGames14681508+43.5
BrokenArXiv 06/2026Math11.152.8+41.7
LAB-Bench LitQA2 (SecureBio)Science54.216.9−37.3
LAB-Bench ProtocolQA (SecureBio)Science86.851.4−35.4
Terminal-Bench 2.1 Extended (Mercor held-out)Code44.411.8−32.6
ProofBench v1.1Math58.026.0−32.0
Terminal-Bench 4.0Code6.137.6+31.5
BrokenArXiv 05/2026Math8.540.0+31.5
AA-Briefcase v1.1 EloAgents11151629+28.2
Toloka Arena (v3)Agents58.031.0−27.0
CyberBench v1.1 OverallCybersecurity43.869.5+25.7
ChartographyMultimodal40.414.7−25.7
SWE-bench Verified Extended (Mercor held-out)Code30.055.3+25.3
Chess Puzzles (task version 2.0)Games66.041.0−25.0
LLM Debate BenchmarkChat & Writing14551620+23.4
ARC-AGI-2General QA84.661.4−23.2
BuseyBench SVGMultimodal6.64.6−20.7
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal20.040.0+20.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal40.020.0−20.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - InsuranceLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.020.0+20.0
FrontierMath Tier 4 (v2) - PrivateMath36.617.1−19.5
LAB-Bench CloningScenarios (SecureBio)Science99.480.1−19.3
AA-LCR Extended (Mercor held-out)Search48.968.2+19.3
FrontierMath Tiers 1-3 (v2) - PrivateMath71.653.0−18.6
LMArena Agent Arena Net ImprovementAgents-1.34.0+17.6
Concept Synth INDUCTION Challenge64 Round 1General QA17.234.4+17.2
MetagenomicsBench V0Science31.748.3+16.6
Roboflow Vision Evals (October 2026, five tasks)Multimodal80.163.5−16.6
DistressBenchSafety69.052.4−16.6
Vibe Code Bench v1.1Code70.486.2+15.8
Bio Capabilities Index (SecureBio)Indexes162141−15.3
Toloka Arena (v1)Agents54.639.3−15.3
Creative Writing v3 EloChat & Writing17202012+14.8
MedXpertQA MM (200-question set)Health & Medicine83.569.0−14.5
BrowseComp Extended (Mercor held-out)Search25.539.5+14.0
GDPval-AA v2.1Professional Work14041718+13.8
Arena Code WebDev — Brand & MarketingCode15531645+13.4
ArXivMath 05/2026Math47.560.8+13.3
Roboflow Vision Evals (September 2026 protocol)Multimodal85.271.9−13.3
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting81.868.6−13.3
SimpleQA Verified (Epoch anti-abstention)General QA69.256.0−13.2
Concept Synth INDUCTION Challenge100 Round 1General QA11.024.0+13.0
LLM Creative Story WritingChat & Writing-0.70.4+12.9
Senior SWE-BenchCode14.727.4+12.7
Finance Agent v2 - Market AnalysisFinance & Accounting74.561.8−12.7
Legal Research BenchLegal34.647.1+12.5
EnterpriseBench: CoreCraftProfessional Work55.968.2+12.3
DAYJOB: FinanceFinance & Accounting2.814.5+11.7
LLM Chess EloGames1107822−11.7
Vision Arena OverallMultimodal12941249−11.5
APEX-SWECode42.053.4+11.4
Tax Agent BenchFinance & Accounting57.769.0+11.3
ChessBench EloGames12651076−11.2
Terminal-Bench 4.0 (Artificial Analysis)Code13.624.7+11.1
HPCT (SecureBio)Science64.753.7−11.0
APEX-Agents 1.1Professional Work79.468.7−10.7
MedXpertQA MM Extended (Mercor held-out)Health & Medicine62.752.0−10.7
Arena Code WebDev — ReactCode15841662+10.6
IOI (Vals v2)Code67.857.7−10.1
Code MigrationCode34.844.8+10.0
FrontierSWE V2Code20.329.5+9.2
Veracode GenAI Code Security (Fall 2026)Cybersecurity60.069.0+9.0
LMArena Agent Arena Net Improvement (Steering Burden)Agents-2.22.6+9.0
MBCT (SecureBio)Science55.046.1−8.9
Arena Code WebDev — FrontendCode15921660+8.7
Finance Agent v2 - AdjustmentsFinance & Accounting54.746.3−8.5
SAGEMath49.240.8−8.4
DAYJOB: HealthcareHealth & Medicine0.08.4+8.4
VIALSMultimodal26.518.2−8.3
Arena Alignment IndexSafety74.482.7+8.3
LMArena Text Arena EloChat & Writing14891442−8.3
Arena Code WebDev — SimulationsCode16181690+8.1
Finance Agent v2 - ComparablesFinance & Accounting50.342.2−8.1
Mystery Game PuzzlesGames37.029.0−8.0
Finance Agent v2 - PrecedentsFinance & Accounting33.825.9−7.9
ArXivMath 06/2026Math62.570.1+7.6
Humanity's Last Exam DiamondGeneral QA31.223.6−7.6
RSI-Exam 0.1 (Full 88)Agents0.30.4+7.5
Artificial Analysis Intelligence Index v4.3.2Indexes39.146.4+7.4
SpeciEval (Pooled English Runs)Safety89.096.4+7.4
SpeciEval (Refusal-Aware Retries, Historical)Safety89.096.4+7.4
LiveBench 2.0Agents67.274.1+6.9
MMMU ProMultimodal84.978.2−6.7
Bug Hunt BenchCode21.027.6+6.7
VCT v2 (SecureBio)Science48.642.0−6.6
ComplexConstraintsChat & Writing42.448.9+6.5
Finance Agent v2 - Financial ModelingFinance & Accounting24.217.7−6.5
LMArena Agent Arena Bash RecoveryAgents-5.47.2+6.3
Arena Code WebDev — GamingCode16381694+6.3
TaxCalcBench TY25Finance & Accounting14.08.0−6.0
Arena Code WebDev — HTMLCode15931636+5.9
Furniture Assembly BenchmarkMultimodal26.720.8−5.8
SM BenchSafety80.686.3+5.7
Arena Code WebDev — Reference-Based DesignCode16081651+5.7
Design Arena Overall Code CategoriesCode13041270−5.6
DeepResearch Bench II (Mercor 130-task set)Search43.449.0+5.6
ChessBench AccuracyGames79.274.0−5.6
DeepSWE v1.1Code65.571.0+5.5
MedScribeHealth & Medicine83.989.4+5.4
ARC-AGI-1 Semi-PrivateGeneral QA95.590.2−5.3
Finance Agent v2 - All PassFinance & Accounting47.542.1−5.3
MineBench — Bradley-Terry RatingGames17651848+5.0
Eval Connections One-Shot with TrapsGames80.085.0+5.0
AA-AnalystAgent (pass^5)Professional Work60.055.0−5.0
RuneBenchGames6.05.7−4.6
Finance Agent v2 - Earnings AnalysisFinance & Accounting79.174.6−4.5
Excel Modeling BenchmarkFinance & Accounting71.367.0−4.3
Arena Code WebDev — Content Creation ToolsCode15741602+4.3
LMArena Agent Arena Confirmed SuccessAgents-3.55.1+4.3
RedlineBenchLegal45.349.5+4.2
Humanity's Last Exam (Text-Only)General QA39.043.1+4.2
GeneBench-Pro Public Release SubsetScience14.010.0−4.0
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine14.718.7+4.0
CadQueryEval (Corrected Geometry)Code84.088.0+4.0
MedCodeHealth & Medicine53.449.6−3.8
CritPt (avg@5)Science14.318.0+3.7
GBENCH One-shot Coding GScoreGames52.656.3+3.7
Vals Index v2.1Indexes51.354.9+3.7
Creative Writing v3 Rubric ScoreChat & Writing15.215.9+3.7
AutomationBench-AA 1.0.6Agents62.065.6+3.5
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting65.368.7+3.4
BrowseComp (Mercor 130-question subset)Search79.275.8−3.4
AIME 2025Math96.7100.0+3.3
CharXiv (descriptive and reasoning combined)Multimodal95.392.0−3.3
Finance Agent v2 - Disclosure AnalysisFinance & Accounting67.764.5−3.3
ChessBench CoherenceGames83.286.3+3.1
FrontierCode 1.1 Extended - ScoreCode56.359.4+3.1
Arena Code WebDev — Data & AnalyticsCode15771556−3.0
AA-LCR v1.1Search83.080.0−3.0
LMArena Agent Arena Praise vs ComplaintAgents-2.93.1+3.0
ACCoRDGeneral QA77.074.0−2.9
FrontierCode 1.1 Extended - Pass RateCode62.365.2+2.9
LegalBenchLegal87.384.4−2.9
Terminal-Bench Science 0.1 (Vals)Science7.110.0+2.9
AA-Omniscience IndexIndexes26.532.0+2.8
LMArena Agent Arena Steering BurdenAgents-7.3-2.3+2.5
Arena Image-to-WebDevCode16001589−2.5
Arena Code WebDev — FullstackCode16291621−2.2
SciCode 1.0.1 (Artificial Analysis)Code59.857.8−2.1
Conceptual Reasoning IndexIndexes64.663.1−2.0
Epoch Capabilities Index (ECI)Indexes158156−1.9
Vals RSI Index v1.1 (four tasks)Agents18.320.2+1.9
APEX-Accounting 1.1Finance & Accounting57.255.3−1.9
SWE-bench VerifiedCode81.079.1−1.9
Harvey HLAB — Criteria Pass RateLegal88.690.5+1.9
ALE-BenchCode18761823−1.9
Artificial Analysis Progress Index v1Indexes317323+1.9
Ramp Accounting BenchFinance & Accounting42.344.1+1.8
MMLU ProGeneral QA90.688.8−1.8
SWE-bench MultilingualCode82.684.2+1.6
APEX-1 (Mercor, mean score)Professional Work67.065.4−1.6
LMArena Agent Arena Bash ReliabilityAgents6.29.3+1.6
ForecastBench Preliminary Dataset Brier IndexForecasting61.760.7−1.0
GDP.pdfProfessional Work23.822.8−1.0
LMCAGeneral QA50.449.4−1.0
Vals Index v2Indexes59.360.2+0.9
GPQA-Bio (SecureBio)Science81.682.5+0.9
CharXiv Extended (Mercor held-out)Multimodal79.178.2−0.9
OTIS Mock AIME 2024-2025Math97.298.1+0.8
DTBenchGeneral QA96.896.0−0.8
Terminal-Bench 2.1Code85.885.0−0.8
ObviousBenchGeneral QA100.099.3−0.7
Arena Code WebDev — Consumer ProductCode15741570−0.6
MMLU-Bio (SecureBio)Science91.490.9−0.5
ProgramBenchCode0.00.5+0.5
WCB (SecureBio)Science58.057.6−0.4
LMArena Agent Arena SteerabilityAgents-0.00.5+0.2
LMArena Agent Arena Tool HallucinationAgents0.71.0+0.2
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.00.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal20.020.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
Medical Long Context Reasoning (MLCR-AA)Health & Medicine15.015.00.0

Gemini 3.7 FlashvsGrok 4.7normalised score marginsData Nov 15, 2026Data: Vals AI, Arena, Mercor and 43 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…