Atlas

Compare Models

Gemini 3.8 Flash vs Grok 4.7

Gemini 3.8 Flash leads by 0.1 capability points (≈95% interval −2.1 to +2.2): statistically tied · P(higher) ≈ 53%

Grok 4.7 wins 171 of 307 shared benchmarks (121 lost, 15 tied)

Capability vs. relative cost
Gemini 3.8 FlashGrok 4.7Closed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Gemini 3.8 FlashvsGrok 4.7capability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins121 – 171 · 307 shared, 15 tied

Comparing matching setups only (37 benchmarks excluded) · suites counted once
← Gemini 3.8 FlashGrok 4.7 →

Gemini 3.8 FlashvsGrok 4.7wins by categoryData Nov 15, 2026Data: Vals AI, Artificial Analysis, Mercor and 73 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Grok 4.7 minus Gemini 3.8 Flash.

CyberGym-E2E-AA (Artificial Analysis)Cybersecurity8.474.0+65.6
AA-PersonalityBench Slop indexChat & Writing71.48.6−63.5
Short Form Bench v1.0Games19402578+63.2
Odysseys2Agents10.768.2+57.5
AA-PersonalityBench Informativeness indexChat & Writing81.033.4−48.1
AA-PersonalityBench Organization indexChat & Writing73.329.6−44.2
SAFE-Teen Health Advice (no prompt)Safety59.116.1+42.9
SAFE-Teen 106: Health Advice (no prompt)Safety59.618.7+40.9
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal40.00.0−40.0
AA-PersonalityBench Transparency indexChat & Writing23.858.1+34.6
LAB-Bench LitQA2 (SecureBio)Science51.516.9−34.6
Vending-Bench 2Games509410537+33.0
TreasuryQA BenchProfessional Work51.018.1−32.9
LAB-Bench ProtocolQA (SecureBio)Science84.051.4−32.6
Terminal-Bench 2.1 Extended (Mercor held-out)Code44.411.8−32.6
BrokenArXiv 06/2026Math20.852.8+32.0
Olam Social Poker EloGames14791508+31.5
SheetBench-50Finance & Accounting45.614.3−31.3
Artificial Analysis Cyber Index v1Indexes25.156.4+31.3
SAFE-Teen 106 worst branch: Health Advice (no prompt)Safety69.039.2+29.8
LLM Chess EloGames1546822−29.7
SAFE-Teen 106 worst branch: Health Advice (under-18 prompt)Safety48.119.2+28.9
Toloka Arena (v3)Agents59.031.0−28.0
ARC-AGI-2General QA89.261.4−27.8
RSI Arena Stage 1 human ratingAgents9811038+27.4
ChessBench EloGames15271076−26.8
ChartographyMultimodal40.914.7−26.2
CyberBench v1.1 OverallCybersecurity43.869.5+25.7
SAFE-Teen Health Advice (under-18 prompt)Safety36.110.6+25.6
SAFE-Teen 106: Health Advice (under-18 prompt)Safety37.512.0+25.5
ARC-AGI-3 (Semi-Private)General QA35.010.0−25.0
Terminal-Bench 4.0Code13.137.6+24.4
AA-Briefcase v1.1 EloAgents12041629+23.3
ProofBench v1.1Math48.026.0−22.0
MetagenomicsBench V0Science26.748.3+21.6
BrokenArXiv 05/2026Math19.040.0+21.0
Extended NYT Connections 940 (Quadratic v1)Games97.476.8−20.6
BuseyBench SVGMultimodal6.64.6−20.6
Concept Synth INDUCTION Challenge64 Round 1General QA14.134.4+20.3
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal20.040.0+20.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal20.00.0−20.0
LAB-Bench CloningScenarios (SecureBio)Science100.080.1−19.9
Code2Video-AA EloCode8991127+18.5
DistressBenchSafety70.752.4−18.3
Mystery Game PuzzlesGames47.029.0−18.0
Biopharma Bench V0.1Health & Medicine46.664.4+17.8
HANDBOOK.md Agents v1.1Professional Work26.544.0+17.5
APEX-SWECode36.453.4+17.0
Chess Puzzles (task version 2.0)Games58.041.0−17.0
Finance Agent v2 - Market AnalysisFinance & Accounting78.561.8−16.7
SAFE-Teen 106: Urgent Safety (no prompt)Safety31.515.1+16.3
Roboflow Vision Evals (October 2026, five tasks)Multimodal79.863.5−16.3
τ³-Banking (Mercor run, bm25_grep retrieval)Agents48.832.6−16.2
CWE-Bench-AA (Artificial Analysis)Cybersecurity52.568.3+15.8
MedXpertQA MM (200-question set)Health & Medicine84.769.0−15.7
GeoBenchMultimodal43073525−15.6
FrontierMath Tiers 1-3 (v2) - PrivateMath68.453.0−15.4
AA-PersonalityBench Warmth indexChat & Writing60.044.8−15.4
SAFE-Teen Urgent Safety (no prompt)Safety31.015.7+15.3
CAD ArenaAgents41.025.8−15.2
LLM Debate BenchmarkChat & Writing15141620+15.0
Concept Synth INDUCTION Challenge100 Round 1General QA9.024.0+15.0
Finance Agent v2 - Financial ModelingFinance & Accounting32.617.7−15.0
SnorkelFinance 2.0Finance & Accounting5.420.2+14.8
SAFE-Teen 106 worst branch: Relationships (no prompt)Safety62.647.9+14.8
Code Migration v1.1Code38.653.2+14.6
Artificial Analysis Coding Agent Index v1.5Indexes41.956.3+14.4
DAYJOB: AccountingFinance & Accounting6.020.0+14.0
CloudOpsBench v1.0Agents42.856.5+13.7
SimpleQA Verified (Epoch anti-abstention)General QA69.756.0−13.7
Creative Writing v3 EloChat & Writing17442012+13.5
Artificial Analysis Coding Agent Index v2.0Indexes33.346.6+13.3
Roboflow Vision Evals (September 2026 protocol)Multimodal85.171.9−13.2
ALE-BenchCode21821823−12.7
AA-LCR Extended (Mercor held-out)Search55.668.2+12.6
DeepsecBench-AA (Artificial Analysis)Cybersecurity14.526.9+12.4
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting80.968.6−12.3
Hermes BenchAgents62.274.4+12.2
GDPval-AA v2.1Professional Work14421718+12.2
SAFE-Teen 106 worst branch: Urgent Safety (no prompt)Safety56.444.4+12.0
DAYJOB: FinanceFinance & Accounting2.814.5+11.7
SnorkelUnderwrite 2.0Finance & Accounting19.130.5+11.4
Vision Arena OverallMultimodal12931249−11.3
MysteryMechanismScience36.525.2−11.3
VIALSMultimodal29.418.2−11.2
Furniture Assembly BenchmarkMultimodal31.720.8−10.8
Bug Hunt BenchCode17.127.6+10.5
ITBench-AAAgents52.542.1−10.5
Finance Agent v2 - PrecedentsFinance & Accounting36.125.9−10.2
FrontierSWE V2Code19.629.5+9.9
Finance Agent v2 - ComparablesFinance & Accounting52.042.2−9.8
EnterpriseBench: CoreCraftProfessional Work58.568.2+9.7
LMArena Text Arena EloChat & Writing14971442−9.7
CritPt v2 (agentic, avg@5)Science65.155.7−9.4
SAFE-Teen Relationships (no prompt)Safety41.732.5+9.1
FrontierCode 2.0 (Artificial Analysis)Code18.727.8+9.0
Bio Capabilities Index (SecureBio)Indexes153141−9.0
SAFE-Teen 106 worst branch: Relationships (under-18 prompt)Safety14.05.1+8.9
MedXpertQA MM Extended (Mercor held-out)Health & Medicine60.852.0−8.8
AA-AnalystAgent (pass^5)Professional Work46.355.0+8.8
HPCT (SecureBio)Science62.453.7−8.7
AML BenchFinance & Accounting57.165.8+8.7
SAFE-Teen 106: Relationships (no prompt)Safety40.131.5+8.6
Finance Agent v2 - AdjustmentsFinance & Accounting54.846.3−8.6
ARC-AGI-1 Semi-PrivateGeneral QA98.590.2−8.3
GameBench 2Games64.072.3+8.3
ChessBench CoherenceGames94.686.3−8.3
Code MigrationCode36.544.8+8.3
RedlineBenchLegal41.349.5+8.2
Legal Research BenchLegal38.947.1+8.2
SAFE-Teen 106 worst branch: Urgent Safety (under-18 prompt)Safety34.826.7+8.1
WitnessBench v2.0Games6.714.8+8.1
Harvey LAB (Mercor run, undisclosed ~213-task cohort)Legal11.419.5+8.1
CadQueryEval (Corrected Geometry)Code96.088.0−8.0
Financial Coaching BenchFinance & Accounting66.374.1+7.8
BrokenArXiv 08/2026 (Native Harnesses)Math27.435.1+7.7
HealthBench (length-adjusted, Artificial Analysis)Health & Medicine48.456.0+7.6
LMArena Code Arena WebDevCode15831641+7.6
ChessBench AccuracyGames81.174.0−7.6
DAYJOB: HealthcareHealth & Medicine0.88.4+7.6
Finance Agent v2 - All PassFinance & Accounting49.742.1−7.6
Vibe Code Bench v1.1Code78.786.2+7.5
SnorkelLegalLegal25.332.8+7.5
BrowseComp (Mercor 130-question subset)Search83.175.8−7.3
BioMysteryBench (Vals Terminus 2)Science62.269.3+7.0
Realm Financial ReasoningFinance & Accounting51.544.5−7.0
Veracode GenAI Code Security (Fall 2026)Cybersecurity62.069.0+7.0
Arena Alignment IndexSafety75.882.7+6.9
Finance Agent v2 - Earnings AnalysisFinance & Accounting81.474.6−6.8
LiveBench 2.0Agents67.374.1+6.8
Visa Agentic Commerce BenchmarkAgents53.360.0+6.7
Medical Long Context Reasoning (MLCR-AA)Health & Medicine21.715.0−6.7
Access to Justice BenchmarkLegal67.173.7+6.6
SpeciEval (Refusal-Aware Retries, Historical)Safety89.996.4+6.5
SpeciEval (Pooled English Runs)Safety89.996.4+6.5
SWE Atlas - Codebase QnACode40.647.0+6.4
MMMU ProMultimodal84.578.2−6.4
HLAB-Vals-1.1Legal12.118.3+6.2
Longform Creative WritingChat & Writing76.883.0+6.2
ArXivMath 08/2026 (Native Harnesses)Math40.446.5+6.1
Blueprint-Bench 2Multimodal0.40.3−6.1
FrontierCode 1.1 Extended - ScoreCode53.559.4+5.9
SnorkelRevOpsProfessional Work5.411.3+5.9
SM BenchSafety80.486.3+5.9
MentalHealthBenchSafety35.541.3+5.8
AA-PersonalityBench Imagination indexChat & Writing41.035.3−5.8
SAGEMath35.140.8+5.7
FrontierCode 1.1 Extended - Pass RateCode59.665.2+5.6
Artificial Analysis Intelligence Index v4.3.2Indexes40.946.4+5.5
Humanity's Last Exam DiamondGeneral QA29.123.6−5.5
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine13.318.7+5.3
DAXBench v2Code87.292.6+5.3
Design Creativity BenchCode60.662.2+5.2
FrontierML ($200 wallet)Agents5.710.9+5.2
Excel Modeling BenchmarkFinance & Accounting72.267.0−5.2
MissionReportProfessional Work54.359.5+5.2
Finance Agent v2.1Finance & Accounting65.860.6−5.2
Terminal-Bench 4.0 (Artificial Analysis)Code19.724.7+5.1
SAFE-Teen Urgent Safety (under-18 prompt)Safety15.210.1+5.0
PLE (Physical-Engineering's Last Exam)Professional Work26.931.9+5.0
APEX-AccountingFinance & Accounting51.746.7−5.0
SnorkelManufacturingProfessional Work7.112.1+5.0
DualEntry Accounting AI BenchmarkFinance & Accounting83.288.1+4.9
MedScribeHealth & Medicine84.589.4+4.9
FrontierMath Tier 4 (v2) - PrivateMath22.017.1−4.9
MKA (Artificial Analysis)Professional Work35.340.1+4.9
SAFE-Teen 106: Urgent Safety (under-18 prompt)Safety14.910.1+4.8
Design Arena Overall Code CategoriesCode12991270−4.8
SWE Atlas v1.1 - OverallCode57.362.1+4.8
Artificial Analysis Intelligence Index v5.0Indexes30.535.3+4.8
ArXivMath 09/2026 (Native Harnesses)Math50.045.3−4.7
AutomationBench-AA 1.0.6Agents60.965.6+4.7
Immigration BenchLegal63.468.0+4.6
Artificial Analysis Intelligence Index v5.0.1Indexes30.835.5+4.6
HoneyBench v0.2 (hack rate)Safety80.585.1−4.6
RuneBenchGames6.05.7−4.6
RSI-Exam 0.1 (Full 88)Agents0.30.4+4.4
APEX-Agents 1.1Professional Work72.968.7−4.2
DAYJOB: TaxFinance & Accounting0.85.0+4.2
FrontierPhysics v0.1Science0.04.2+4.2
CVDP (517-problem set)Code50.754.9+4.2
ACCoRDGeneral QA78.274.0−4.1
Finance Agent v2 - Disclosure AnalysisFinance & Accounting68.564.5−4.0
AA-LCR v1.1Search84.080.0−4.0
GDP.xlsxProfessional Work18.322.3+4.0
Next.js AI Agent EvaluationsAgents100.096.0−4.0
Snorkel WorkplaceAgentsProfessional Work11.215.2+4.0
LMArena Agent Arena Bash RecoveryAgents-0.77.2+3.9
AutoResearchExam (hidden-test AUARC)Agents0.50.5+3.9
sudo L7Code11.715.6+3.9
MatFlowBench Pass@1Science18.922.8+3.9
AA-PersonalityBench Expressiveness indexChat & Writing79.175.3−3.8
Conceptual Reasoning IndexIndexes65.763.1−3.5
LMCAGeneral QA52.949.4−3.5
CharXiv (descriptive and reasoning combined)Multimodal95.492.0−3.4
Physical-Engineering's Last Exam (hidden test)Professional Work26.830.2+3.4
APEX-1 (Mercor, mean score)Professional Work68.865.4−3.4
AIME 2025Math96.7100.0+3.3
Ivo-micro1 ContractBenchLegal55.058.3+3.3
LLM Creative Story WritingChat & Writing0.10.4+3.2
VCT v2 (SecureBio)Science45.242.0−3.2
Eval Connections One-Shot with TrapsGames88.085.0−3.0
BioSimBench 1.0Science15.312.3−3.0
HealthBenchHealth & Medicine57.154.2−3.0
SAFE-Teen Relationships (under-18 prompt)Safety4.01.1+2.9
Vals Legal IndexIndexes69.872.7+2.9
Terminal-Bench Science 0.1 (Vals)Science8.611.4+2.9
MBCT (SecureBio)Science48.946.1−2.8
ArXivMath 06/2026Math67.470.1+2.8
FeatureArc (whole-feature chain gate)Code23.326.0+2.7
DeepSWE v1.1Code73.771.0−2.7
SAFE-Teen 106: Relationships (under-18 prompt)Safety3.71.0+2.7
Vals Index v2.2Indexes56.058.6+2.7
AA-3D v1.0 EloCode974992+2.7
Terminal-Bench 2.1Code87.685.0−2.6
LegalBenchLegal87.084.4−2.6
MineBench — Bradley-Terry RatingGames18061848+2.5
ArXivMath 05/2026Math58.360.8+2.5
Westworld Finance Diligence BenchFinance & Accounting49.747.2−2.5
Snorkel Digital Work IndexIndexes16.919.3+2.4
GBENCH One-shot Coding GScoreGames58.656.3−2.4
Clipping Bench v1.0Games596384−2.3
Tax Agent BenchFinance & Accounting66.869.0+2.2
Epoch Capabilities Index (ECI)Indexes159156−2.1
SWE-bench Multimodal (Vals v1, 480 Tasks)Code40.842.9+2.1
Vals Index v2Indexes62.360.2−2.0
BrowseComp Extended (Mercor held-out)Search41.539.5−2.0
TaxCalcBench TY25Finance & Accounting6.08.0+2.0
AA-PersonalityBench Conviction indexChat & Writing84.882.9−1.9
MMLU ProGeneral QA90.788.8−1.9
LMArena Agent Arena SteerabilityAgents4.30.5−1.9
LMArena Agent Arena Confirmed SuccessAgents1.35.1+1.9
Creative Writing v3 Rubric ScoreChat & Writing15.615.9+1.8
Legacy-Bench (Fireworks, avg@3)Code52.354.0+1.7
Ramp Accounting BenchFinance & Accounting45.744.1−1.6
Vals Finance IndexIndexes60.558.8−1.6
SWE-bench MultilingualCode85.884.2−1.6
SWE-bench VerifiedCode80.679.1−1.5
ViBench 2.0Code0.01.5+1.5
MedCodeHealth & Medicine48.149.6+1.4
Excel Modeling Benchmark v1.1Finance & Accounting72.971.5−1.4
SWE-bench Verified Extended (Mercor held-out)Code54.055.3+1.3
AA-Omniscience IndexIndexes29.632.0+1.2
Hermes IndexIndexes38.139.3+1.2
SciCode 1.0.1 (Artificial Analysis)Code56.657.8+1.2
Arena Image-to-WebDevCode15941589−1.1
Humanity's Last Exam (Text-Only)General QA42.143.1+1.0
GeneBench-Pro Public Release SubsetScience11.010.0−1.0
WellnessBenchSafety47.048.0+1.0
BrokenArXiv 09/2026 (Native Harnesses)Math26.727.7+0.9
LMArena Agent Arena Praise vs ComplaintAgents4.93.1−0.9
Vals Science IndexIndexes35.834.8−0.9
SWE-TogetherCode64.265.1+0.9
LMArena Agent Arena Net ImprovementAgents3.84.0+0.9
Artificial Analysis Progress Index v1Indexes320323+0.8
OTIS Mock AIME 2024-2025Math98.998.1−0.8
RiemannBench (November 2026 rescoring)Math56.857.6+0.8
LegalReasonBenchLegal39.640.4+0.8
IOI (Vals v2)Code56.957.7+0.8
CharXiv Extended (Mercor held-out)Multimodal78.978.2−0.7
Snorkel Agentic Coding 2.0Code32.631.9−0.7
ObviousBenchGeneral QA100.099.3−0.7
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting68.168.7+0.7
WMDP-Bio (SecureBio)Science89.488.8−0.6
LMArena Agent Arena Steering BurdenAgents-1.3-2.3−0.5
ComplexConstraintsChat & Writing48.448.9+0.5
ProgramBenchCode1.00.5−0.5
WCB (SecureBio)Science57.157.6+0.5
LMArena Agent Arena Net Improvement (Steering Burden)Agents2.32.6+0.5
Terminal-Bench-Science 0.1Science10.09.5−0.5
SAGE v1.1Math88.288.7+0.5
GDP.pdfProfessional Work23.222.8−0.4
Public Benefits Bench v1.1Professional Work65.365.6+0.3
ATLAS-Finance v1.0Finance & Accounting4.75.0+0.3
Harvey HLAB — Criteria Pass RateLegal90.290.5+0.3
LMArena Agent Arena Bash ReliabilityAgents9.99.3−0.3
CritPt (avg@5)Science18.318.0−0.3
DTBenchGeneral QA95.796.0+0.3
LMArena Agent Arena Tool HallucinationAgents0.41.0+0.3
Vals RSI Index v1.1 (four tasks)Agents19.920.2+0.3
Snorkel SWE-bench CLICode6.36.1−0.2
ForecastBench Preliminary Dataset Brier IndexForecasting60.560.7+0.2
BenchBench-CloningScience10.810.9+0.1
Vals Index v2.1Indexes54.954.9+0.1
Dual-Arm Sim (task completion)Agents0.00.00.0
FrontierMath ErdősMath0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal20.020.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal20.020.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0

Gemini 3.8 FlashvsGrok 4.7normalised score marginsData Nov 15, 2026Data: Vals AI, Artificial Analysis, Mercor and 73 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…