Atlas

Compare Models

Gemini 3.7 Flash vs GPT-6 Sol

GPT-6 Sol leads by 4.4 capability points (≈95% interval +2.3 to +6.5): a clear lead over Gemini 3.7 Flash · P(higher) ≈ 100%

Wins 103 of 185 shared benchmarks (60 lost, 22 tied)

Capability vs. relative cost
Gemini 3.7 FlashGPT-6 SolClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Gemini 3.7 FlashvsGPT-6 Solcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins60 – 103 · 185 shared, 22 tied

Comparing matching setups only (56 benchmarks excluded) · suites counted once
← Gemini 3.7 FlashGPT-6 Sol →

Gemini 3.7 FlashvsGPT-6 Solwins by categoryData Nov 15, 2026Data: Vals AI, Mercor, Artificial Analysis and 36 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6 Sol minus Gemini 3.7 Flash.

BrokenArXiv 06/2026Math11.195.4+84.3
Olam Social Poker EloGames14681542+80.4
BrokenArXiv 05/2026Math8.587.0+78.5
FrontierMath Tier 4 (v2) - PrivateMath36.690.0+53.4
ArXivMath 05/2026Math47.590.0+42.5
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal40.00.0−40.0
Terminal-Bench 4.0Code6.143.8+37.7
BrowseComp Extended (Mercor held-out)Search25.563.2+37.7
CyberBench v1.1 OverallCybersecurity43.878.0+34.2
LMArena Agent Arena Net ImprovementAgents-1.38.8+33.3
Furniture Assembly BenchmarkMultimodal26.758.3+31.7
HieroglyphBenchMultimodal55.026.0−29.0
ArXivMath 06/2026Math62.591.0+28.5
ALE-BenchCode18762637+26.9
RiemannBenchMath39.264.8+25.6
ProofBench v1.1Math58.083.0+25.0
Terminal-Bench Science 0.1 (Vals)Science7.130.0+22.9
Code MigrationCode34.857.2+22.4
AA-LCR Extended (Mercor held-out)Search48.970.2+21.3
Creative Writing v3 EloChat & Writing17202126+20.6
Harvey's Legal Agent Benchmark - Banking FinanceLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - InsuranceLegal20.00.0−20.0
Mystery Game PuzzlesGames37.056.0+19.0
FrontierMath Tiers 1-3 (v2) - PrivateMath71.689.8+18.2
LLM Chess EloGames11071545+18.0
GBENCH One-shot Coding GScoreGames52.670.1+17.5
Vibe Code Bench v1.1Code70.487.8+17.4
LMArena Agent Arena Net Improvement (Steering Burden)Agents-2.26.9+17.2
BioSecBench-Refusal V1 Clean RerunsSafety54.837.9−16.9
CritPt (avg@5)Science14.330.9+16.6
Finance Agent v2 - Market AnalysisFinance & Accounting74.558.0−16.6
CadQueryEval (Corrected Geometry)Code84.0100.0+16.0
MMMU-Pro Extended (Mercor held-out)Multimodal36.451.9+15.5
IOI (Vals v2)Code67.882.6+14.8
LiveBench 2.0Agents67.281.9+14.7
SM BenchSafety80.666.0−14.6
Humanity's Last Exam DiamondGeneral QA31.245.2+14.0
TaxCalcBench TY25Finance & Accounting14.028.0+14.0
ChartographyMultimodal40.453.6+13.2
Arena Alignment IndexSafety74.487.6+13.2
Veracode GenAI Code Security (Fall 2026)Cybersecurity60.073.0+13.0
EnterpriseBench: CoreCraftProfessional Work55.943.1−12.8
Terminal-Bench 4.0 (Artificial Analysis)Code13.626.3+12.6
Finance Agent v2 - Disclosure AnalysisFinance & Accounting67.755.6−12.1
GeneBench-Pro Public Release SubsetScience14.026.0+12.0
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting81.870.0−11.8
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting65.353.6−11.7
Finance Agent v2 - AdjustmentsFinance & Accounting54.743.2−11.5
Finance Agent v2 - All PassFinance & Accounting47.536.0−11.5
ScBench V1 Clean RerunsScience50.461.5+11.1
APEX-Agents 1.1Professional Work79.468.7−10.7
MetagenomicsBench V1Science31.742.0+10.3
MetagenomicsBench V0Science31.742.0+10.3
BrowseComp (Mercor 130-question subset)Search79.289.4+10.2
Wiz Cyber Model Arena v3.9Cybersecurity59.169.3+10.2
Chess Puzzles (task version 2.0)Games66.056.0−10.0
ScBench-Long V1 Clean RerunsScience18.828.4+9.7
Finance Agent v2 - ComparablesFinance & Accounting50.340.8−9.5
SpeciEval (Pooled English Runs)Safety89.098.5+9.4
SpeciEval (Refusal-Aware Retries, Historical)Safety89.098.5+9.4
SWE-bench Verified Extended (Mercor held-out)Code30.039.3+9.3
Eval Connections One-Shot with TrapsGames80.089.0+9.0
Conceptual Reasoning IndexIndexes64.671.0+8.9
BuseyBench SVGMultimodal6.67.5+8.9
LMArena Agent Arena Steering BurdenAgents-7.310.1+8.7
LMCAGeneral QA50.459.1+8.7
Humanity's Sixth SenseMultimodal39.831.2−8.6
Artificial Analysis Intelligence Index v4.3.2Indexes39.147.6+8.6
SimpleQA Verified (Epoch anti-abstention)General QA69.260.7−8.5
SpatialBench V1 Clean RerunsScience59.167.5+8.4
ComplexConstraintsChat & Writing42.450.8+8.4
AA-Briefcase v1.1 EloAgents11151266+8.3
TxBench Preclinical Pharmacology V1 Clean RerunsScience55.363.0+7.7
VariantBench V1 Clean RerunsScience31.438.7+7.3
Humanity's Last Exam (Text-Only)General QA39.046.3+7.3
GPQA-Bio (SecureBio)Science81.688.9+7.3
LMArena Agent Arena SteerabilityAgents-0.014.5+7.3
Finance Agent v2 - Earnings AnalysisFinance & Accounting79.172.4−6.7
TxBench Antibody Discovery V1 Clean RerunsScience43.537.0−6.6
DAYJOB: FinanceFinance & Accounting2.89.3+6.5
LMArena Agent Arena Praise vs ComplaintAgents-2.910.0+6.4
MedCodeHealth & Medicine53.447.1−6.3
NoCoT-Bench NCRI 15.2General QA126115−6.3
BioSecBench-Surveillance V1 Clean RerunsScience41.547.7+6.3
Vals Index v2.1Indexes51.357.5+6.3
LMArena Agent Arena Bash RecoveryAgents-5.47.0+6.2
MBCT (SecureBio)Science55.061.1+6.1
Artificial Analysis Progress Index v1Indexes317334+5.8
Legal Research BenchLegal34.628.8−5.8
LAB-Bench ProtocolQA (SecureBio)Science86.881.1−5.7
LMArena Text Arena EloChat & Writing14891457−5.7
BioSecBench-Function V1 Clean RerunsScience33.739.3+5.6
ACCoRDGeneral QA77.082.4+5.4
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine14.720.0+5.3
Finance Agent v2 - PrecedentsFinance & Accounting33.828.5−5.3
MedXpertQA MM Extended (Mercor held-out)Health & Medicine62.768.0+5.3
Harvey HLAB — Criteria Pass RateLegal88.683.5−5.1
ARC-AGI-2General QA84.689.6+5.0
ChessBench CoherenceGames83.288.2+5.0
TxBench Oligo Discovery V1 Clean RerunsScience35.640.6+5.0
Finance Agent v2 - Financial ModelingFinance & Accounting24.219.3−4.9
CharXiv Extended (Mercor held-out)Multimodal79.183.8+4.7
VCT (SecureBio)Science49.354.0+4.7
Tax Agent BenchFinance & Accounting57.753.0−4.6
Epoch Capabilities Index (ECI)Indexes158164+4.5
SAGEMath49.244.8−4.4
Creative Writing v3 Rubric ScoreChat & Writing15.216.1+4.2
LMArena Agent Arena Confirmed SuccessAgents-3.54.5+4.0
DAYJOB: HealthcareHealth & Medicine0.03.6+3.6
APEX-1 (Mercor, mean score)Professional Work67.063.6−3.4
AIME 2025Math96.7100.0+3.3
Vals Index v2Indexes59.362.6+3.3
GlobeBenchGeneral QA21.825.0+3.2
Roboflow Vision Evals (October 2026, five tasks)Multimodal80.176.9−3.2
HPCT (SecureBio)Science64.767.8+3.1
APEX-SWECode42.045.0+3.0
Roboflow Vision Evals (September 2026 protocol)Multimodal85.282.3−2.9
OTIS Mock AIME 2024-2025Math97.2100.0+2.8
WCB (SecureBio)Science58.060.5+2.5
APEX-Accounting 1.1Finance & Accounting57.259.7+2.5
LAB-Bench LitQA2 (SecureBio)Science54.251.8−2.4
LMArena Agent Arena Bash ReliabilityAgents6.210.9+2.4
MMMU ProMultimodal84.982.5−2.3
ChessBench EloGames12651303+2.3
MedXpertQA MM (200-question set)Health & Medicine83.581.3−2.2
SciCode 1.0.1 (Artificial Analysis)Code59.857.6−2.2
ProgramBenchCode0.02.0+2.0
Terminal-Bench 2.1 Extended (Mercor held-out)Code44.442.4−2.0
ChessBench AccuracyGames79.277.4−1.9
MedScribeHealth & Medicine83.982.0−1.9
Design Arena Overall Code CategoriesCode13041293−1.8
SpatialBench-Long V1 Clean RerunsScience29.831.6+1.8
GDPval (Mercor 213-task rubric score)Professional Work85.283.5−1.7
MMLU ProGeneral QA90.688.9−1.7
NoCoT-Bench NCKI (kspine v3)General QA121119−1.7
GDP.pdf (Artificial Analysis, All-pass)Professional Work23.625.2+1.6
FACTS Benchmark SuiteGeneral QA71.272.5+1.3
Medical Long Context Reasoning (MLCR-AA)Health & Medicine15.016.1+1.1
DeepSWE v1.1Code65.566.6+1.1
DistressBenchSafety69.068.0−1.0
GDPval Extended (Mercor held-out rubric score)Professional Work57.858.7+0.9
LAB-Bench CloningScenarios (SecureBio)Science99.498.5−0.9
ForecastBench Preliminary Dataset Brier IndexForecasting61.762.6+0.9
Gray Swan IPI Attack Success (k=15)Safety9.210.1−0.9
SWE-bench MultilingualCode82.681.7−0.9
SWE-bench VerifiedCode81.080.3−0.7
VCT v2 (SecureBio)Science48.649.3+0.7
ObviousBenchGeneral QA100.099.3−0.7
AA-LCR v1.1Search83.083.7+0.7
MMLU-Bio (SecureBio)Science91.490.8−0.6
DTBenchGeneral QA96.897.3+0.5
Bio Capabilities Index (SecureBio)Indexes162161−0.5
DeepResearch Bench II (Mercor 130-task set)Search43.442.9−0.5
Terminal-Bench 2.1Code85.885.4−0.4
AutomationBench-AA 1.0.6Agents62.061.7−0.3
AA-Omniscience IndexIndexes26.527.1+0.3
CharXiv (descriptive and reasoning combined)Multimodal95.395.0−0.3
Excel Modeling BenchmarkFinance & Accounting71.371.5+0.2
LMArena Agent Arena Tool HallucinationAgents0.71.0+0.2
GDPval-AA v2.1Professional Work14041402−0.1
ARC-AGI-1 Semi-PrivateGeneral QA95.595.50.0
EpiBench V1 Clean RerunsScience22.022.00.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal0.00.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.00.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.00.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
SUPERChemScience72.872.80.0
Toloka Arena (v3)Agents58.058.00.0

Gemini 3.7 FlashvsGPT-6 Solnormalised score marginsData Nov 15, 2026Data: Vals AI, Mercor, Artificial Analysis and 36 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…