Atlas

Compare Models

Gemini 3.7 Flash vs Grok 4.6

Grok 4.6 leads by 0.6 capability points (≈95% interval −1.3 to +2.6): statistically tied · P(higher) ≈ 73%

Wins 118 of 244 shared benchmarks (107 lost, 19 tied)

Capability vs. relative cost
Gemini 3.7 FlashGrok 4.6Closed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Gemini 3.7 FlashvsGrok 4.6capability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins107 – 118 · 244 shared, 19 tied

Comparing matching setups only (20 benchmarks excluded) · suites counted once
← Gemini 3.7 FlashGrok 4.6 →

Gemini 3.7 FlashvsGrok 4.6wins by categoryData Nov 15, 2026Data: Vals AI, Arena, Mercor and 52 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Grok 4.6 minus Gemini 3.7 Flash.

SWE-bench Verified Extended (Mercor held-out)Code30.078.0+48.0
Gray Swan IPI Attack Success (k=15)Safety9.251.8−42.6
MRCR v2 (8-needle, Context Arena) - 128K pointwiseGeneral QA88.649.0−39.6
UnityBench-Mini3DCode31.270.6+39.4
Antidote: Everyday EditionSafety1075966−38.9
HieroglyphBenchMultimodal55.020.3−34.7
Document ArenaProfessional Work14761438−34.3
Hemingway-benchChat & Writing1105983−33.8
Olam Social Poker EloGames14681499+33.7
ClockBench AccuracyMultimodal53.320.0−33.3
Chess Puzzles (task version 2.0)Games66.033.0−33.0
LLM Chess EloGames1107443−27.3
LAB-Bench LitQA2 (SecureBio)Science54.228.9−25.3
LLM Creative Story WritingChat & Writing-0.7-2.9−24.6
AA-Briefcase EloAgents11161534+24.3
CyberBench v1.1 OverallCybersecurity43.867.7+23.9
ChessBench EloGames1265891−22.2
MRCR v2 (8-needle, Context Arena) - 128K AUCGeneral QA93.971.9−22.0
NoCoT-Bench NCRI 15.2General QA12686.8−21.6
LLM Debate BenchmarkChat & Writing14551604+21.2
Roboflow Vision Evals (October 2026, five tasks)Multimodal80.159.1−21.0
SAGEMath49.228.9−20.3
BuseyBench SVGMultimodal6.64.6−20.3
IOI (Vals v2)Code67.847.6−20.2
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - InsuranceLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.020.0+20.0
SimpleQA Verified (Epoch anti-abstention)General QA69.249.3−19.9
ChartographyMultimodal40.420.5−19.9
AA-AnalystAgent (pass^5)Professional Work60.041.3−18.7
Concept Synth INDUCTION Challenge64 Round 1General QA17.235.9+18.8
LAB-Bench ProtocolQA (SecureBio)Science86.868.1−18.7
Bio Capabilities Index (SecureBio)Indexes162137−18.3
ARC-AGI-2General QA84.667.1−17.5
Shroombench (mushroom identification, top-1)Multimodal61.044.0−17.0
HANDBOOK.mdProfessional Work10.827.3+16.5
Roboflow Vision Evals (September 2026 protocol)Multimodal85.268.7−16.5
AA-LCR Extended (Mercor held-out)Search48.965.2+16.3
𝜏³-BankingAgents35.550.7+15.3
SWE-bench VerifiedCode80.895.6+14.8
APEX-SWECode42.056.4+14.4
MedXpertQA MM Extended (Mercor held-out)Health & Medicine62.748.4−14.3
HLE Extended (Mercor held-out)General QA31.645.8+14.2
ChessBench CoherenceGames83.269.2−14.0
Concept Synth INDUCTION Challenge100 Round 1General QA11.025.0+14.0
Tax Agent BenchFinance & Accounting57.771.4+13.7
Legal Research BenchLegal34.648.1+13.5
Furniture Assembly BenchmarkMultimodal26.740.0+13.3
Wiz Cyber Model Arena v3.9Cybersecurity59.746.6−13.1
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting65.377.8+12.5
Finance Agent v2 - Market AnalysisFinance & Accounting74.562.2−12.3
ChessBench AccuracyGames79.267.8−12.2
CadQueryEval (Corrected Geometry)Code84.072.0−12.0
Finance Agent v2 - PrecedentsFinance & Accounting33.821.9−11.9
Senior SWE-BenchCode14.726.3+11.6
Terminal-Bench 4.0Code6.117.2+11.1
Eval Connections One-Shot with TrapsGames80.091.0+11.0
MMMU-Pro Extended (Mercor held-out)Multimodal36.426.0−10.4
BioSecBench-Function V1 Clean RerunsScience33.744.0+10.4
GDPval-AA v2Professional Work14331643+10.3
FrontierChallenge (Pass Rate)Science10.320.6+10.3
SkillsBench with SkillsAgents65.955.8−10.1
SWE-bench MultilingualCode82.692.7+10.1
DAYJOB: FinanceFinance & Accounting2.812.8+10.0
Arena Code WebDev — Brand & MarketingCode15531621+10.0
GDPval-AA v2.1Professional Work14041627+9.8
Code MigrationCode34.844.6+9.8
EnterpriseBench: CoreCraftProfessional Work55.965.6+9.7
Humanity's Last Exam DiamondGeneral QA31.221.5−9.7
Finance Agent v2 - ComparablesFinance & Accounting50.341.2−9.1
VariantBench V1 Clean RerunsScience31.440.4+9.0
ScBench V1 Clean RerunsScience50.459.1+8.7
MedCodeHealth & Medicine53.444.7−8.7
Excel Modeling BenchmarkFinance & Accounting71.362.7−8.6
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting81.873.2−8.6
SM BenchSafety80.689.0+8.4
MedXpertQA MM (200-question set)Health & Medicine83.575.2−8.3
SpatialBench V1 Clean RerunsScience59.167.3+8.1
LMArena Agent Arena Net ImprovementAgents-1.31.1+8.0
ARC-AGI-1 Semi-PrivateGeneral QA95.587.5−8.0
WordleBenchGames97.089.0−8.0
BioSecBench-Refusal V1 Clean RerunsSafety54.846.9−7.9
Vision Arena OverallMultimodal12941263−7.8
TxBench Oligo Discovery V1 Clean RerunsScience35.643.3+7.8
LiveBench 2.0Agents67.274.9+7.8
WCB (SecureBio)Science58.050.3−7.7
Terminal-Bench 4.0 (Artificial Analysis)Code13.621.2+7.6
Vals RSI Index v1.1 (four tasks)Agents18.325.5+7.2
SUPERChemScience72.865.7−7.1
HLE-Bio (SecureBio)Science36.229.2−7.0
MetagenomicsBench V0Science31.738.7+7.0
MetagenomicsBench V1Science31.738.7+7.0
ProofBench v1.1Math58.051.0−7.0
SkillsBench without SkillsAgents44.938.2−6.7
BabaIsBench Stage 1: The LakeGames93.386.7−6.7
GDP.pdfProfessional Work23.817.2−6.6
NoCoT-Bench NCKI (kspine v3)General QA121112−6.5
Arena Code WebDev — ReactCode15841631+6.4
GlobeBenchGeneral QA21.828.1+6.4
Bug Hunt BenchCode21.027.3+6.3
TxBench Preclinical Pharmacology V1 Clean RerunsScience55.361.7+6.3
Arena Alignment IndexSafety74.480.8+6.3
BioSecBench-Surveillance V1 Clean RerunsScience41.547.5+6.1
BioSecBench-Surveillance V0 (102 tasks)Science41.547.5+6.0
VIALSMultimodal26.520.5−6.0
ComplexConstraintsChat & Writing42.436.5−5.9
Vibe Code Bench v1.1Code70.476.2+5.8
RSI-Exam 0.1 (Full 88)Agents0.30.4+5.8
GPQA-Bio (SecureBio)Science81.687.4+5.8
LMArena Text Arena EloChat & Writing14891456−5.7
FrontierMath Tiers 1-3 (v2) - PrivateMath71.666.0−5.6
LMArena Agent Arena Bash RecoveryAgents-5.45.7+5.6
AHK-EvalCode91.797.2+5.6
SpatialBench V0 VerifiedScience64.369.9+5.5
CritPt (avg@5)Science14.319.7+5.4
ACCoRDGeneral QA77.071.6−5.4
Artificial Analysis Intelligence Index v4.3.2Indexes39.144.3+5.3
Finance Agent v2 - Financial ModelingFinance & Accounting24.218.9−5.2
LAB-Bench CloningScenarios (SecureBio)Science99.494.2−5.2
Finance Agent v2 - Earnings AnalysisFinance & Accounting79.173.9−5.2
Finance Agent v2 - AdjustmentsFinance & Accounting54.749.6−5.1
FrontierCode 1.1 Extended - ScoreCode56.361.3+5.0
FrontierSWE V2Code20.325.3+5.0
Artificial Analysis Intelligence Index v4.3Indexes39.444.4+5.0
AutomationBench-AA 1.0.6Agents62.067.0+5.0
RuneBenchGames6.06.3+4.9
Toloka Arena (v1)Agents54.649.7−4.9
FrontierMath Tier 4 (v2) - PrivateMath36.631.7−4.9
BrowseComp (Mercor 130-question subset)Search79.284.0+4.8
Arena Code WebDev — FrontendCode15921629+4.8
FrontierCode 1.1 Extended - Pass RateCode62.367.0+4.7
MBCT (SecureBio)Science55.050.4−4.6
Finance Agent v2 - All PassFinance & Accounting47.542.9−4.5
TxBench Oligo Discovery V0Science39.544.0+4.4
MMMU ProMultimodal84.980.5−4.4
Terminal-Bench Science 0.1 (Vals)Science7.111.4+4.3
Arena Code WebDev — GamingCode16381674+4.0
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine14.718.7+4.0
BabaIsBench Stage 0: The IntroGames100.096.0−4.0
BrowseComp Extended (Mercor held-out)Search25.521.5−4.0
GDPval Extended (Mercor held-out rubric score)Professional Work57.861.8+4.0
HealthBench Professional (Length-Adjusted)Health & Medicine48.444.4−4.0
Humanity's Last Exam (Text-Only)General QA39.042.9+3.9
Harvey HLAB — Criteria Pass RateLegal88.692.5+3.9
LMArena Agent Arena Net Improvement (Steering Burden)Agents-2.2-0.2+3.8
TaxEval v2Finance & Accounting74.771.1−3.6
SpatialBench-Long V1 Clean RerunsScience29.826.2−3.6
Arena Code WebDev — Reference-Based DesignCode16081635+3.6
Arena Code WebDev — SimulationsCode16181648+3.4
Finance Agent v2 - Disclosure AnalysisFinance & Accounting67.764.4−3.4
SciCode 1.0.1 (Artificial Analysis)Code59.856.5−3.4
Arena Code WebDev — FullstackCode16291616−3.4
AIME 2025Math96.7100.0+3.3
RedlineBenchLegal45.348.5+3.2
HPCT (SecureBio)Science64.761.6−3.1
APEX-Accounting 1.1Finance & Accounting57.260.3+3.1
SpatialBench-Long V0 VerifiedScience22.719.7−3.0
Mystery Game PuzzlesGames37.034.0−3.0
Conceptual Reasoning IndexIndexes64.662.5−2.8
DAYJOB: HealthcareHealth & Medicine0.02.8+2.8
LMArena Agent Arena Steering BurdenAgents-7.3-1.8+2.8
MineBench — Bradley-Terry RatingGames17651811+2.8
Medical Long Context Reasoning (MLCR-AA)Health & Medicine15.012.2−2.8
GBENCH One-shot Coding GScoreGames52.655.4+2.8
Design Arena Overall Code CategoriesCode13041288−2.7
Terminal-Bench 2.1Code85.888.4+2.6
MedScribeHealth & Medicine83.986.5+2.6
MortgageTax (Vals all 1,258 documents)Finance & Accounting66.764.2−2.5
CharXiv (descriptive and reasoning combined)Multimodal95.392.9−2.4
ScBench-Long V1 Clean RerunsScience18.816.5−2.3
Arena Code WebDev — Data & AnalyticsCode15771562−2.3
APEX-Agents 1.1Professional Work79.477.3−2.1
EnterpriseOps-Gym-AAAgents50.448.3−2.1
Veracode GenAI Code Security (Fall 2026)Cybersecurity60.058.0−2.0
AA-LCR v1.1Search83.081.0−2.0
AA-Omniscience IndexIndexes26.530.5+2.0
Toloka Arena (v3)Agents58.056.0−2.0
DeepSWE v1.1Code65.567.5+2.0
OTIS Mock AIME 2024-2025Math97.299.2+1.9
ALE-BenchCode18761822−1.9
LMCAGeneral QA50.448.5−1.9
LMArena Agent Arena Bash ReliabilityAgents6.210.0+1.9
BioSecBench-Function V0Science40.142.0+1.9
TxBench Antibody Discovery V0Science50.051.9+1.8
SpeciEval (Pooled English Runs)Safety89.090.8+1.8
CharXiv Extended (Mercor held-out)Multimodal79.177.3−1.8
SpeciEval (Refusal-Aware Retries, Historical)Safety89.090.8+1.8
LMArena Agent Arena SteerabilityAgents-0.03.3+1.6
CyberArenaCybersecurity67.165.5−1.6
EpiBench V1 Clean RerunsScience22.023.6+1.6
GPQA DiamondGeneral QA95.593.9−1.5
Arena Image-to-WebDevCode16001607+1.5
Artificial Analysis Progress Index v1Indexes317322+1.5
ATLAS-Finance v1.1Finance & Accounting3.34.7+1.3
MMLU ProGeneral QA90.689.4−1.2
Arena Code WebDev — Consumer ProductCode15741583+1.1
Terminal-Bench 2.1 Extended (Mercor held-out)Code44.445.5+1.1
ProgramBenchCode0.01.0+1.0
VCT (SecureBio)Science49.348.3−1.0
GeneBench-Pro Public Release SubsetScience14.015.0+1.0
TxBench Antibody Discovery V1 Clean RerunsScience43.544.5+1.0
LegalBenchLegal87.386.3−0.9
Arena Code WebDev — HTMLCode15931600+0.9
Vals Index v2.1Indexes51.352.1+0.8
HackerRank-BenchCode28.527.7−0.8
GDPval (Mercor 213-task rubric score)Professional Work85.286.0+0.8
APEX-1 (Mercor, mean score)Professional Work67.067.7+0.7
ForecastBench Preliminary Dataset Brier IndexForecasting61.761.0−0.7
PG-LLM-50Science0.30.3−0.5
DTBenchGeneral QA96.897.3+0.5
LMArena Agent Arena Confirmed SuccessAgents-3.5-2.6+0.4
Epoch Capabilities Index (ECI)Indexes158158−0.4
LiveCodeBench (Vals AI public dataset)Code88.788.2−0.4
ForecastBench Tournament Overall Brier IndexForecasting66.265.8−0.4
MMLU-Bio (SecureBio)Science91.491.8+0.4
Arena Code WebDev — Content Creation ToolsCode15741577+0.4
xbench-ScienceQAScience70.270.4+0.2
LMArena Agent Arena Praise vs ComplaintAgents-2.9-2.5+0.2
LMArena Agent Arena Tool HallucinationAgents0.71.0+0.2
Vals Index v2Indexes59.359.2−0.1
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal20.020.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal20.020.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal40.040.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal20.020.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
MirrorCode (ML, +Private, 2L)Code41.141.10.0
ObviousBenchGeneral QA100.0100.00.0
ScBench-Long V0 VerifiedScience24.224.20.0

Gemini 3.7 FlashvsGrok 4.6normalised score marginsData Nov 15, 2026Data: Vals AI, Arena, Mercor and 52 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…