Atlas

Compare Models

Gemini 3.7 Flash vs GPT-6.1 Sol

GPT-6.1 Sol leads by 6.9 capability points (≈95% interval +4.6 to +9.2): a clear lead over Gemini 3.7 Flash · P(higher) ≈ 100%

Wins 122 of 175 shared benchmarks (35 lost, 18 tied)

Capability vs. relative cost
Gemini 3.7 FlashGPT-6.1 SolClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Gemini 3.7 FlashvsGPT-6.1 Solcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins35 – 122 · 175 shared, 18 tied

Comparing matching setups only (53 benchmarks excluded) · suites counted once
← Gemini 3.7 FlashGPT-6.1 Sol →

Gemini 3.7 FlashvsGPT-6.1 Solwins by categoryData Nov 15, 2026Data: Vals AI, Mercor, Center for AI Safety and 33 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6.1 Sol minus Gemini 3.7 Flash.

BrokenArXiv 06/2026Math11.1100.0+88.9
Olam Social Poker EloGames14681543+81.5
BrokenArXiv 05/2026Math8.590.0+81.5
Concept Synth INDUCTION Challenge100 Round 1General QA11.090.0+79.0
Concept Synth INDUCTION Challenge64 Round 1General QA17.295.3+78.1
FrontierMath Tier 4 (v2) - PrivateMath36.6100.0+63.4
Furniture Assembly BenchmarkMultimodal26.780.0+53.3
ArXivMath 05/2026Math47.595.0+47.5
Terminal-Bench 4.0Code6.153.4+47.3
Terminal-Bench Science 0.1 (Vals)Science7.152.9+45.7
MASKSafety53.798.3+44.6
Mystery Game PuzzlesGames37.080.0+43.0
BrowseComp Extended (Mercor held-out)Search25.567.7+42.2
ProofBench v1.1Math58.099.0+41.0
ALE-BenchCode18762990+39.4
Terminal-Bench 4.0 (Artificial Analysis)Code13.651.5+37.9
ArXivMath 06/2026Math62.594.4+31.9
Code MigrationCode34.865.1+30.3
IOI (Vals v2)Code67.896.9+29.1
HPCT Refusal (CAIS)Safety13.042.0+29.0
LMArena Agent Arena Net Improvement (Steering Burden)Agents-2.212.5+27.7
Humanity's Last Exam DiamondGeneral QA31.258.8+27.6
GBENCH One-shot Coding GScoreGames52.679.0+26.4
EnigmaEvalGeneral QA33.959.5+25.6
MMMU-Pro Extended (Mercor held-out)Multimodal36.461.3+24.9
TaxCalcBench TY25Finance & Accounting14.038.0+24.0
HieroglyphBenchMultimodal55.031.4−23.6
VIALSMultimodal26.549.1+22.6
FrontierMath Tiers 1-3 (v2) - PrivateMath71.693.7+22.1
LLM Chess EloGames11071612+20.7
Machiavelli Deception (CAIS)Safety85.578.8+20.3
Harvey's Legal Agent Benchmark - Banking FinanceLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal40.020.0−20.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - InsuranceLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.020.0+20.0
Agent Red Teaming k=100 (CAIS)Safety38.518.8+19.7
AA-Briefcase v1.1 EloAgents11151469+19.4
GeneBench-Pro Public Release SubsetScience14.033.0+19.0
Medical Long Context Reasoning (MLCR-AA)Health & Medicine15.033.9+18.9
TextQuests Game ProgressGames52.971.4+18.5
Vibe Code Bench v1.1Code70.488.9+18.5
LiveBench 2.0Agents67.285.4+18.2
ChessBench EloGames12651562+17.7
CritPt (avg@5)Science14.331.7+17.4
BioSecBench-Refusal V1 Clean RerunsSafety54.837.6−17.2
TextQuests Average Cumulative HarmSafety24.429.3−16.7
BrowseComp (Mercor 130-question subset)Search79.295.4+16.2
SWE-bench Verified Extended (Mercor held-out)Code30.045.3+15.3
DAYJOB: FinanceFinance & Accounting2.818.0+15.2
LMArena Agent Arena Praise vs ComplaintAgents-2.927.5+15.2
SpatialViz-BenchMultimodal75.390.3+15.0
TxBench Oligo Discovery V1 (113 evaluations)Science37.852.5+14.8
VariantBench V1 Clean RerunsScience31.446.0+14.7
MetagenomicsBench V1Science31.745.7+14.0
Arena Alignment IndexSafety74.487.9+13.5
ScBench V1 Clean RerunsScience50.463.6+13.2
BuseyBench SVGMultimodal6.68.0+13.1
Artificial Analysis Intelligence Index v4.3.2Indexes39.151.8+12.8
NoCoT-Bench NCRI 15.2General QA126149+12.6
LMCAGeneral QA50.462.8+12.4
BioSecBench-Function V1 Clean RerunsScience33.746.0+12.4
SM BenchSafety80.669.0−11.7
Wiz Cyber Model Arena v3.9Cybersecurity59.170.7+11.6
VCT-Refusal (CAIS)Safety4.816.1+11.3
GDP.pdfProfessional Work23.835.0+11.2
Conceptual Reasoning IndexIndexes64.672.7+11.2
Artificial Analysis Progress Index v1Indexes317348+10.9
LMArena Agent Arena Steering BurdenAgents-7.313.4+10.4
SpatialBench-Long V1 Clean RerunsScience29.839.9+10.1
Finance Agent v2 - Market AnalysisFinance & Accounting74.564.5−10.0
Finance Agent v2 - AdjustmentsFinance & Accounting54.744.7−10.0
AA-AnalystAgent (pass^5)Professional Work60.050.0−10.0
Vals Index v2.1Indexes51.361.2+9.9
DeepSWE v1.1Code65.575.2+9.7
ARC-AGI-2General QA84.694.2+9.6
Finance Agent v2 - Disclosure AnalysisFinance & Accounting67.758.4−9.3
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine14.724.0+9.3
BioSecBench-Surveillance V1 Clean RerunsScience41.550.8+9.3
Design Arena Overall Code CategoriesCode13041356+8.6
Epoch Capabilities Index (ECI)Indexes158169+8.4
SpeciEval (Pooled English Runs)Safety89.097.4+8.3
SpeciEval (Refusal-Aware Retries, Historical)Safety89.097.4+8.3
SpatialBench V1 Clean RerunsScience59.167.3+8.1
Finance Agent v2 - PrecedentsFinance & Accounting33.825.8−8.0
Toloka Arena (v3)Agents58.066.0+8.0
ScBench-Long V1 Clean RerunsScience18.826.7+8.0
PCT Political Inconsistency (CAIS)Safety45.137.3+7.8
Finance Agent v2 - ComparablesFinance & Accounting50.342.6−7.7
DAYJOB: HealthcareHealth & Medicine0.07.6+7.6
LMArena Agent Arena Confirmed SuccessAgents-3.511.7+7.6
Finance Agent v2 - Earnings AnalysisFinance & Accounting79.171.5−7.6
Finance Agent v2 - All PassFinance & Accounting47.539.9−7.5
AA-Omniscience IndexIndexes26.541.5+7.5
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting81.874.5−7.3
APEX-Accounting 1.1Finance & Accounting57.264.3+7.1
Humanity's Sixth SenseMultimodal39.846.6+6.8
SWE-bench VerifiedCode81.087.3+6.3
EpiBench V1 Clean RerunsScience22.028.3+6.3
APEX-Agents 1.1Professional Work79.473.2−6.2
CharXiv Extended (Mercor held-out)Multimodal79.185.1+6.0
NoCoT-Bench NCKI (kspine v3)General QA121130+5.9
Humanity's Last ExamGeneral QA45.651.3+5.7
EnterpriseBench: CoreCraftProfessional Work55.950.3−5.6
DistressBenchSafety69.074.2+5.2
GDPval-AA v2.1Professional Work14041521+5.2
SUPERChemScience72.877.8+5.0
LMArena Agent Arena Bash RecoveryAgents-5.44.5+5.0
APEX-SWECode42.046.9+4.9
SimpleQA Verified (Epoch anti-abstention)General QA69.273.9+4.7
Tax Agent BenchFinance & Accounting57.762.3+4.6
MedCodeHealth & Medicine53.448.8−4.6
AutomationBench-AA 1.0.6Agents62.066.6+4.5
CyberBench v1.1 OverallCybersecurity43.839.3−4.5
TxBench Preclinical Pharmacology V1 Clean RerunsScience55.359.7+4.3
ChessBench CoherenceGames83.287.5+4.3
IntPhys 2Multimodal68.272.3+4.1
Ramp Accounting BenchFinance & Accounting42.346.4+4.1
SciCode 1.0.1 (Artificial Analysis)Code59.855.8−4.1
FACTS Benchmark SuiteGeneral QA71.275.3+4.0
ACCoRDGeneral QA77.080.9+3.9
Legal Research BenchLegal34.638.5+3.8
AIME 2025Math96.7100.0+3.3
ARC-AGI-1 Semi-PrivateGeneral QA95.598.5+3.0
ProgramBenchCode0.03.0+3.0
OTIS Mock AIME 2024-2025Math97.2100.0+2.8
SAGEMath49.246.5−2.7
MedScribeHealth & Medicine83.986.5+2.5
SWE-bench MultilingualCode82.684.9+2.3
MedXpertQA MM (200-question set)Health & Medicine83.585.7+2.2
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting65.363.2−2.1
Chess Puzzles (task version 2.0)Games66.068.0+2.0
MedXpertQA MM Extended (Mercor held-out)Health & Medicine62.764.7+2.0
ChessBench AccuracyGames79.280.7+1.6
Roboflow Vision Evals (October 2026, five tasks)Multimodal80.181.7+1.6
Terminal-Bench 2.1 Extended (Mercor held-out)Code44.445.8+1.4
LMArena Text Arena EloChat & Writing14891482−1.3
Finance Agent v2 - Financial ModelingFinance & Accounting24.223.1−1.1
AA-LCR v1.1Search83.084.0+1.0
TxBench Antibody Discovery V1 (100 evaluations)Science50.749.7−1.0
Harvey HLAB — Criteria Pass RateLegal88.687.8−0.8
MindCube TinyMultimodal88.489.1+0.7
CharXiv (descriptive and reasoning combined)Multimodal95.396.0+0.7
LMArena Agent Arena Bash ReliabilityAgents6.25.0−0.6
Gray Swan IPI Attack Success (k=15)Safety9.28.6+0.6
Excel Modeling BenchmarkFinance & Accounting71.370.8−0.6
ERQAMultimodal77.778.2+0.5
MMLU ProGeneral QA90.691.0+0.4
Terminal-Bench 2.1Code85.885.4−0.4
Vision Arena OverallMultimodal12941293−0.3
Roboflow Vision Evals (September 2026 protocol)Multimodal85.285.5+0.3
DTBenchGeneral QA96.896.5−0.3
MMMU ProMultimodal84.985.1+0.2
LMArena Agent Arena Tool HallucinationAgents0.71.0+0.2
ForecastBench Preliminary Dataset Brier IndexForecasting61.761.70.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.00.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.00.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
ObviousBenchGeneral QA100.0100.00.0

Gemini 3.7 FlashvsGPT-6.1 Solnormalised score marginsData Nov 15, 2026Data: Vals AI, Mercor, Center for AI Safety and 33 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…