Atlas

Compare Models

Gemini 3.7 Flash vs GPT-5.6 Sol

GPT-5.6 Sol leads by 3.5 capability points (≈95% interval +1.4 to +5.5): a clear lead over Gemini 3.7 Flash · P(higher) ≈ 100%

Wins 134 of 257 shared benchmarks (101 lost, 22 tied)

Capability vs. relative cost
Gemini 3.7 FlashGPT-5.6 SolClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Gemini 3.7 FlashvsGPT-5.6 Solcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins101 – 134 · 257 shared, 22 tied

Comparing matching setups only (66 benchmarks excluded) · suites counted once
← Gemini 3.7 FlashGPT-5.6 Sol →

Gemini 3.7 FlashvsGPT-5.6 Solwins by categoryData Nov 15, 2026Data: Vals AI, benchmarks.bio, Artificial Analysis and 62 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-5.6 Sol minus Gemini 3.7 Flash.

Olam Social Poker EloGames14681534+71.7
Odysseys2Agents5.862.0+56.2
BrokenArXiv 06/2026Math11.167.3+56.2
FrontierMath Tier 4 (v2) - PrivateMath36.682.9+46.3
MASKSafety53.795.4+41.7
Concept Synth INDUCTION Challenge64 Round 1General QA17.257.8+40.6
LLM Creative Story WritingChat & Writing-0.72.5+35.9
RiemannBenchMath39.274.4+35.2
CyberBench v1.1 OverallCybersecurity43.876.3+32.6
Concept Synth INDUCTION Challenge100 Round 1General QA11.043.0+32.0
Terminal-Bench 4.0Code6.137.3+31.2
UnityBench-Mini3DCode31.262.3+31.1
Furniture Assembly BenchmarkMultimodal26.756.7+30.0
ChessBench — Historical Output RepairGames70.742.7−28.0
HieroglyphBenchMultimodal55.027.6−27.4
LLM Debate BenchmarkChat & Writing14551645+27.0
ArXivMath 06/2026Math62.588.5+26.0
ProofBench v1.1Math58.083.0+25.0
LMArena Agent Arena Net ImprovementAgents-1.36.3+25.0
TextQuests Average Cumulative HarmSafety24.431.4−23.7
IOI (Vals v2)Code67.891.2+23.3
MirrorCode (ML, +Private, 2L)Code41.120.0−21.1
Mystery Game PuzzlesGames37.058.0+21.0
Senior SWE-BenchCode14.734.7+20.0
HardGeoBench <=10miMultimodal46.326.3−20.0
HardGeoBench Country AccuracyMultimodal67.547.5−20.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal40.020.0−20.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - InsuranceLegal20.00.0−20.0
ScBench-Long V0 VerifiedScience24.243.9+19.7
SM BenchSafety80.661.1−19.5
Shroombench (mushroom identification, top-1)Multimodal61.042.0−19.0
ALE-BenchCode18762410+18.9
LLM Chess EloGames11071550+18.2
Code MigrationCode34.852.9+18.1
Gray Swan IPI Attack Success (k=15)Safety9.227.0−17.8
FrontierMath Tiers 1-3 (v2) - PrivateMath71.689.1+17.5
Antidote: Everyday EditionSafety10751027−17.1
HardGeoBench <=100miMultimodal56.340.0−16.2
TxBench Antibody Discovery V0Science50.033.8−16.2
TaxCalcBench TY25Finance & Accounting14.030.0+16.0
SWE-bench VerifiedCode80.896.2+15.4
GBENCH One-shot Coding GScoreGames52.667.7+15.1
HardGeoBench Avg DistanceMultimodal8951381−15.0
CritPt (avg@5)Science14.328.6+14.3
AA-Briefcase EloAgents11161361+14.3
SpatialBench-Long V0 VerifiedScience22.736.4+13.6
Legal Research BenchLegal34.648.1+13.5
Hemingway-benchChat & Writing11051057−13.3
SimpleVQA with toolsMultimodal77.864.7−13.1
LMArena Agent Arena Net Improvement (Steering Burden)Agents-2.24.6+12.9
Terminal-Bench Science 0.1 (Vals)Science7.120.0+12.9
Creative Writing v3 EloChat & Writing17201974+12.8
HANDBOOK.mdProfessional Work10.823.5+12.7
BioTIER RefusalSafety84.471.9−12.5
AA-AnalystAgent (pass^5)Professional Work60.047.5−12.5
TxBench Antibody Discovery V1 Clean RerunsScience43.531.4−12.2
HealthBench Professional (Length-Adjusted)Health & Medicine48.460.5+12.1
FrontierSWE V2Code20.332.2+11.9
SkillsBench with SkillsAgents65.954.1−11.8
HPCT (SecureBio)Science64.776.3+11.6
MindCube TinyMultimodal88.476.8−11.6
BrowseComp (Mercor 130-question subset)Search79.290.6+11.4
Medical Long Context Reasoning (MLCR-AA)Health & Medicine15.026.1+11.1
HPCT Refusal (CAIS)Safety13.024.0+11.0
ScBench-Long V1 Clean RerunsScience18.829.6+10.8
P2NPCBench (NP-complete, pass@1)Code57.868.4+10.6
Finance Agent v2 - Market AnalysisFinance & Accounting74.564.2−10.4
Tax Agent BenchFinance & Accounting57.768.0+10.3
SpatialBench V0 VerifiedScience64.374.5+10.1
Vibe Code Bench v1.1Code70.480.5+10.1
SpeciEval (Pooled English Runs)Safety89.099.0+10.0
SpeciEval (Refusal-Aware Retries, Historical)Safety89.099.0+10.0
Veracode GenAI Code Security (Fall 2026)Cybersecurity60.070.0+10.0
GeneBench-Pro Public Release SubsetScience14.024.0+10.0
HackerRank-BenchCode28.538.4+10.0
Humanity's Sixth SenseMultimodal39.830.0−9.8
Arena Alignment IndexSafety74.484.2+9.7
EnterpriseBench: CoreCraftProfessional Work55.946.2−9.7
MedCodeHealth & Medicine53.444.0−9.4
ScBench V1 Clean RerunsScience50.459.8+9.4
NoCoT-Bench NCRI 15.2General QA126109−9.1
LiveBench 2.0Agents67.276.3+9.1
SpatialBench V1 Clean RerunsScience59.168.1+9.0
𝜏³-BankingAgents35.544.3+8.9
Machiavelli Deception (CAIS)Safety85.582.7+8.5
LMCAGeneral QA50.458.5+8.1
ComplexConstraintsChat & Writing42.450.5+8.1
DeepResearch Bench II (Mercor 130-task set)Search43.451.5+8.1
Harvey HLAB — Criteria Pass RateLegal88.680.5−8.0
BabyVisionMultimodal73.565.5−8.0
ARC-AGI-2General QA84.692.5+7.9
Artificial Analysis Intelligence Index v4.3.2Indexes39.147.0+7.9
SpatialBench-Long V1 Clean RerunsScience29.837.5+7.7
Roboflow Vision Evals (October 2026, five tasks)Multimodal80.172.4−7.7
Artificial Analysis Intelligence Index v4.3Indexes39.447.1+7.6
Finance Agent v2 - PrecedentsFinance & Accounting33.826.2−7.6
BioMysteryBench - Human SolvableScience87.179.5−7.6
EnterpriseOps-Gym-AAAgents50.442.9−7.5
Wiz Cyber Model Arena v3.9Cybersecurity59.766.9+7.2
Terminal-Bench 4.0 (Artificial Analysis)Code13.620.7+7.1
Chess Puzzles (task version 2.0)Games66.059.0−7.0
Terminal-Bench 2.1 Extended (Mercor held-out)Code44.437.4−7.0
Toloka Arena (v1)Agents54.647.6−7.0
GDP.pdfProfessional Work23.830.7+6.9
BrowseComp Extended (Mercor held-out)Search25.532.3+6.8
Vals Index v2.1Indexes51.358.0+6.7
ERQAMultimodal77.771.2−6.5
VCT-Refusal (CAIS)Safety4.811.3+6.5
AutoformBenchMath52.558.9+6.5
LMArena Agent Arena Steering BurdenAgents-7.35.5+6.4
WCB (SecureBio)Science58.064.4+6.4
Roboflow Vision Evals (September 2026 protocol)Multimodal85.279.0−6.2
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting81.875.6−6.2
MBCT (SecureBio)Science55.061.1+6.1
Finance Agent v2 - All PassFinance & Accounting47.541.4−6.1
LiveCodeBench (Vals AI public dataset)Code88.782.6−6.0
APEX-SWECode42.048.0+6.0
GDP.pdf - all-pass rate reported by GoogleProfessional Work34.040.0+6.0
TxBench Preclinical Pharmacology V1 Clean RerunsScience55.361.3+6.0
ChessBench CoherenceGames83.289.1+5.9
BioSecBench-Surveillance V1 Clean RerunsScience41.547.3+5.8
Conceptual Reasoning IndexIndexes64.668.7+5.7
Finance Agent v2 - AdjustmentsFinance & Accounting54.749.2−5.5
SpatialViz-BenchMultimodal75.380.7+5.4
BuseyBench SVGMultimodal6.66.1−5.3
TxBench Oligo Discovery V0Science39.534.5−5.0
VCT (SecureBio)Science49.354.3+5.0
Finance Agent v2 - ComparablesFinance & Accounting50.345.3−5.0
RedlineBenchLegal45.350.3+5.0
HyperBrowseCompSearch31.726.7−5.0
Artificial Analysis Progress Index v1Indexes317331+4.9
Bio Capabilities Index (SecureBio)Indexes162169+4.9
LAB-Bench ProtocolQA (SecureBio)Science86.881.9−4.9
Creative Writing v3 Rubric ScoreChat & Writing15.216.2+4.8
GDPval-AA v2.1Professional Work14041511+4.7
Finance Agent v2 - Earnings AnalysisFinance & Accounting79.174.4−4.7
DistressBenchSafety69.073.6+4.6
LMArena Agent Arena Praise vs ComplaintAgents-2.96.3+4.6
ChartographyMultimodal40.445.0+4.6
GDPval-AA v2Professional Work14331524+4.5
DeepSWE v1.1Code65.561.1−4.4
Finance Agent v2 - Disclosure AnalysisFinance & Accounting67.763.3−4.4
Vals Index v2Indexes59.363.7+4.4
FrontierCode 1.1 Extended - ScoreCode56.360.6+4.3
IntPhys 2Multimodal68.263.9−4.3
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine14.718.7+4.0
CadQueryEval (Corrected Geometry)Code84.088.0+4.0
VariantBench V1 Clean RerunsScience31.435.3+4.0
MRCR v2 (8-needle, Context Arena) - 128K pointwiseGeneral QA88.692.4+3.9
EpiBench V1 Clean RerunsScience22.025.8+3.8
Epoch Capabilities Index (ECI)Indexes158163+3.7
GPQA-Bio (SecureBio)Science81.685.3+3.7
ObviousBenchGeneral QA100.096.5−3.5
Finance Agent v2 - Financial ModelingFinance & Accounting24.220.7−3.4
PCT Political Inconsistency (CAIS)Safety45.141.7+3.4
SAGEMath49.252.6+3.3
PG-LLM-50Science0.30.4+3.3
LMArena Agent Arena SteerabilityAgents-0.06.5+3.3
MRCR v2 (8-needle, Context Arena) - 128K AUCGeneral QA93.997.1+3.2
Document ArenaProfessional Work14761480+3.2
LMArena Agent Arena Bash RecoveryAgents-5.40.7+3.1
ChessBench AccuracyGames79.276.4−3.0
Toloka Arena (v3)Agents58.061.0+3.0
Eval Connections One-Shot with TrapsGames80.083.0+3.0
LMArena Agent Arena Confirmed SuccessAgents-3.52.3+2.9
MMLU Pro - LawGeneral QA81.979.1−2.8
ChessBench EloGames12651218−2.8
OTIS Mock AIME 2024-2025Math97.2100.0+2.8
AHK-EvalCode91.794.4+2.8
DAYJOB: FinanceFinance & Accounting2.85.5+2.7
LMArena Agent Arena Bash ReliabilityAgents6.211.4+2.6
Continual Learning Exam (Trial 1)Agents25.327.8+2.5
AA-Omniscience IndexIndexes26.522.0−2.3
MMMU ProMultimodal84.982.7−2.2
MMLU Pro - HistoryGeneral QA84.081.9−2.1
SciCode 1.0.1 (Artificial Analysis)Code59.857.8−2.1
ARC-AGI-1 Semi-PrivateGeneral QA95.597.5+2.0
AutomationBench-AA 1.0.6Agents62.060.1−1.9
GlobeBenchGeneral QA21.823.7+1.9
MMLU Pro - PsychologyGeneral QA91.189.2−1.9
BioSecBench-Function V1 Clean RerunsScience33.735.5+1.9
Agent Red Teaming k=100 (CAIS)Safety38.540.3−1.9
BioSecBench-Function V0Science40.138.3−1.8
SUPERChemScience72.871.0−1.8
DAYJOB: HealthcareHealth & Medicine0.01.6+1.6
SkillsBench without SkillsAgents44.943.3−1.6
MMLU Pro - PhysicsGeneral QA94.392.8−1.5
ProgramBenchCode0.01.5+1.5
Terminal-Bench 2.1Code85.887.3+1.5
Vision Arena OverallMultimodal12941288−1.5
MMLU Pro - EngineeringGeneral QA89.387.8−1.4
LAB-Bench LitQA2 (SecureBio)Science54.252.8−1.4
Learn2Play BenchGames58.357.0−1.3
MedScribeHealth & Medicine83.985.2+1.3
MMLU Pro - Computer ScienceGeneral QA92.090.7−1.2
BioMysteryBench - Human DifficultScience43.544.7+1.2
ACCoRDGeneral QA77.075.9−1.1
DTBenchGeneral QA96.895.7−1.1
MMLU Pro - EconomicsGeneral QA91.790.6−1.1
MMLU Pro (Vals 5-Shot CoT)General QA90.189.1−1.0
Excel Modeling BenchmarkFinance & Accounting71.372.3+1.0
MMLU Pro - PhilosophyGeneral QA89.488.4−1.0
AA-LCR v1.1Search83.084.0+1.0
CharXiv Reasoning (with tools)Multimodal88.787.8−0.9
TextQuests Game ProgressGames52.952.1−0.8
VCT v2 (SecureBio)Science48.649.4+0.8
Humanity's Last Exam DiamondGeneral QA31.230.4−0.8
LMArena Text Arena EloChat & Writing14891485−0.8
MMLU Pro - MathMath96.095.3−0.7
ATLAS-Finance v1.1Finance & Accounting3.34.0+0.7
MortgageTax (Vals all 1,258 documents)Finance & Accounting66.767.3+0.6
MMLU Pro - HealthGeneral QA84.784.1−0.6
LAB-Bench CloningScenarios (SecureBio)Science99.498.8−0.6
TxBench Oligo Discovery V1 Clean RerunsScience35.635.0−0.6
MMLU Pro - BusinessGeneral QA92.893.3+0.5
GPQA DiamondGeneral QA95.594.9−0.5
Agents' Last Exam - release-reported pass rate, subset unspecifiedAgents26.326.7+0.4
MetagenomicsBench V0Science31.731.3−0.4
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting65.364.9−0.4
MMLU Pro - ChemistryGeneral QA92.392.0−0.4
NoCoT-Bench NCKI (kspine v3)General QA121121−0.3
MetagenomicsBench V1Science31.731.3−0.3
MMLU-Bio (SecureBio)Science91.491.1−0.3
ForecastBench Preliminary Dataset Brier IndexForecasting61.761.4−0.3
LegalBenchLegal87.387.0−0.3
LMArena Agent Arena Tool HallucinationAgents0.71.0+0.2
MMLU Pro - BiologyGeneral QA94.794.6−0.1
MMLU Pro - OtherGeneral QA87.687.7+0.1
ForecastBench Tournament Overall Brier IndexForecasting66.266.1−0.1
Humanity's Last ExamGeneral QA45.645.5−0.1
TaxEval v2Finance & Accounting74.774.80.0
BabaIsBench Stage 0: The IntroGames100.0100.00.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal0.00.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.00.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.00.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
LABBench2Science82.182.10.0
VIALSMultimodal26.526.50.0
WordleBenchGames97.097.00.0

Gemini 3.7 FlashvsGPT-5.6 Solnormalised score marginsData Nov 15, 2026Data: Vals AI, benchmarks.bio, Artificial Analysis and 62 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…