Atlas

Compare Models

Gemini 3.8 Flash vs GPT-6 Sol

GPT-6 Sol leads by 3.1 capability points (≈95% interval +0.8 to +5.4): a clear lead over Gemini 3.8 Flash · P(higher) ≈ 100%

Wins 132 of 223 shared benchmarks (90 lost, 1 tied)

Capability vs. relative cost
Gemini 3.8 FlashGPT-6 SolClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Gemini 3.8 FlashvsGPT-6 Solcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins90 – 132 · 223 shared, 1 tied

Comparing matching setups only (72 benchmarks excluded) · suites counted once
← Gemini 3.8 FlashGPT-6 Sol →

Gemini 3.8 FlashvsGPT-6 Solwins by categoryData Nov 15, 2026Data: Artificial Analysis, Vals AI, Mercor and 53 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6 Sol minus Gemini 3.8 Flash.

BrokenArXiv 06/2026Math20.895.4+74.5
AA-PersonalityBench Slop indexChat & Writing71.41.0−71.2
AA-PersonalityBench Informativeness indexChat & Writing81.010.5−71.2
Olam Social Poker EloGames14791542+68.5
FrontierMath Tier 4 (v2) - PrivateMath22.090.0+68.0
BrokenArXiv 05/2026Math19.087.0+68.0
AA-PersonalityBench Organization indexChat & Writing73.312.4−61.5
Vending-Bench 2Games509414428+56.5
ArXivMath 08/2026 (Native Harnesses)Math40.493.0+52.6
BrokenArXiv 08/2026 (Native Harnesses)Math27.478.6+51.2
AA-PersonalityBench Expressiveness indexChat & Writing79.133.4−46.2
Short Form Bench v1.0Games19402394+45.0
ProofBench v1.1Math48.083.0+35.0
AA-PersonalityBench Conviction indexChat & Writing84.850.5−34.6
CyberBench v1.1 OverallCybersecurity43.878.0+34.2
DeepsecBench-AA (Artificial Analysis)Cybersecurity14.546.3+31.8
ArXivMath 05/2026Math58.390.0+31.7
Terminal-Bench 4.0Code19.749.4+29.7
ARC-AGI-3 (Semi-Private)General QA35.05.9−29.1
HieroglyphBenchMultimodal54.626.0−28.6
ScBench V1 Clean RerunsScience34.061.5+27.5
Furniture Assembly BenchmarkMultimodal31.758.3+26.7
IOI (Vals v2)Code56.982.6+25.7
ArXivMath 06/2026Math67.491.0+23.6
Terminal-Bench Science 0.1 (Vals)Science8.631.4+22.9
AA-3D v1.0 EloCode9741131+22.8
TaxCalcBench TY25Finance & Accounting6.028.0+22.0
BrowseComp Extended (Mercor held-out)Search41.563.2+21.7
FrontierMath Tiers 1-3 (v2) - PrivateMath68.489.8+21.4
Code MigrationCode36.557.2+20.6
Finance Agent v2 - Market AnalysisFinance & Accounting78.558.0−20.5
Creative Writing v3 EloChat & Writing17442126+19.3
MentalHealthBenchSafety35.553.9+18.4
FrontierCode 2.0 (Artificial Analysis)Code18.735.6+16.8
LMArena Agent Arena Net ImprovementAgents3.88.8+16.6
ALE-BenchCode21822637+16.1
Humanity's Last Exam DiamondGeneral QA29.145.2+16.1
CritPt v2 (agentic, avg@5)Science65.180.9+15.7
MMMU-Pro Extended (Mercor held-out)Multimodal36.451.9+15.5
GameBench 2Games64.079.4+15.5
EnterpriseBench: CoreCraftProfessional Work58.543.1−15.4
AA-PersonalityBench Transparency indexChat & Writing23.839.1+15.4
MetagenomicsBench V1Science26.742.0+15.3
MetagenomicsBench V0Science26.742.0+15.3
GeneBench-Pro Public Release SubsetScience11.026.0+15.0
SWE-bench Verified Extended (Mercor held-out)Code54.039.3−14.7
AA-LCR Extended (Mercor held-out)Search55.670.2+14.6
LiveBench 2.0Agents67.381.9+14.6
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting68.153.6−14.5
SM BenchSafety80.466.0−14.5
Tax Agent BenchFinance & Accounting66.853.0−13.7
Finance Agent v2 - All PassFinance & Accounting49.736.0−13.7
RiemannBenchMath51.264.8+13.6
Finance Agent v2 - Financial ModelingFinance & Accounting32.619.3−13.3
ChessBench EloGames15271303−13.3
Artificial Analysis Intelligence Index v5.0.1Indexes30.844.2+13.3
Finance Agent v2 - Disclosure AnalysisFinance & Accounting68.555.6−12.8
ChartographyMultimodal40.953.6+12.7
BioMysteryBench (Vals Terminus 2)Science62.274.8+12.6
CritPt (avg@5)Science18.330.9+12.6
MBCT (SecureBio)Science48.961.1+12.2
Arena Alignment IndexSafety75.887.6+11.8
Artificial Analysis Cyber Index v1Indexes25.136.8+11.7
CWE-Bench-AA (Artificial Analysis)Cybersecurity52.564.2+11.7
Finance Agent v2 - AdjustmentsFinance & Accounting54.843.2−11.6
AA-PersonalityBench Imagination indexChat & Writing41.029.6−11.5
GBENCH One-shot Coding GScoreGames58.670.1+11.4
ChessBenchGames73.362.0−11.3
Hermes BenchAgents62.273.5+11.3
Finance Agent v2 - ComparablesFinance & Accounting52.040.8−11.2
Veracode GenAI Code Security (Fall 2026)Cybersecurity62.073.0+11.0
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting80.970.0−10.8
Humanity's Sixth SenseMultimodal41.631.2−10.4
Professional Reasoning Benchmark - LegalLegal48.938.6−10.3
Legal Research BenchLegal38.928.8−10.1
SAGEMath35.144.8+9.7
Decision-Bench v1.1Professional Work26.936.5+9.6
BioSecBench-Refusal V1 Clean RerunsSafety47.437.9−9.5
SimpleBenchGeneral QA82.473.1−9.3
Vals Science IndexIndexes35.845.0+9.2
Vibe Code Bench v1.1Code78.787.8+9.2
Finance Agent v2 - Earnings AnalysisFinance & Accounting81.472.4−9.0
Mystery Game PuzzlesGames47.056.0+9.0
SimpleQA Verified (Epoch anti-abstention)General QA69.760.7−9.0
WellnessBenchSafety47.056.0+9.0
BuseyBench SVGMultimodal6.67.5+9.0
HealthBenchHealth & Medicine57.148.2−8.9
LMArena Agent Arena Net Improvement (Steering Burden)Agents2.36.9+8.7
RCM-Bench-1Health & Medicine90.281.5−8.7
Public Benefits Bench v1.1Professional Work65.356.6−8.7
SpeciEval (Pooled English Runs)Safety89.998.5+8.6
SpeciEval (Refusal-Aware Retries, Historical)Safety89.998.5+8.6
APEX-SWECode36.445.0+8.6
ScBench-Long V1 Clean RerunsScience19.928.4+8.5
CyberGym-E2E-AA (Artificial Analysis)Cybersecurity8.40.0−8.4
Harvey's Legal Agent BenchmarkLegal10.01.7−8.3
TxBench Preclinical Pharmacology V1 Clean RerunsScience54.763.0+8.3
SpatialBench V1 Clean RerunsScience59.467.5+8.1
Quant-Arena (iterative total score)Finance & Accounting43.051.1+8.1
LegalReasonBenchLegal39.631.9−7.7
Finance Agent v2 - PrecedentsFinance & Accounting36.128.5−7.6
TxBench Oligo Discovery V1 Clean RerunsScience33.140.6+7.5
Conceptual Reasoning IndexIndexes65.771.0+7.4
VariantBench V1 Clean RerunsScience31.438.7+7.3
Extended NYT Connections 940 (Quadratic v1)Games97.490.1−7.3
Harvey LAB (Mercor run, undisclosed ~213-task cohort)Legal11.44.1−7.3
MedXpertQA MM Extended (Mercor held-out)Health & Medicine60.868.0+7.2
Terminal-Bench-Science 0.1Science10.017.1+7.1
LMArena Text Arena EloChat & Writing14971457−7.1
VCT (SecureBio)Science46.954.0+7.1
DualEntry Accounting AI BenchmarkFinance & Accounting83.276.2−7.0
Clipping Bench v1.0Games5961240+6.9
Artificial Analysis Intelligence Index v4.3.2Indexes40.947.6+6.7
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine13.320.0+6.7
Terminal-Bench 4.0 (Artificial Analysis)Code19.726.3+6.6
DAYJOB: FinanceFinance & Accounting2.89.3+6.5
ChessBench CoherenceGames94.688.2−6.4
TxBench Antibody Discovery V1 Clean RerunsScience43.337.0−6.3
MysteryMechanismScience36.530.2−6.3
BrowseComp (Mercor 130-question subset)Search83.189.4+6.3
LMCAGeneral QA52.959.1+6.2
NoCoT-Bench NCRI 15.2General QA126115−6.1
Longform Creative WritingChat & Writing76.882.8+6.0
Hermes IndexIndexes38.144.1+6.0
τ³-Banking (Mercor run, bm25_grep retrieval)Agents48.843.0−5.8
HealthBench (length-adjusted, Artificial Analysis)Health & Medicine48.454.2+5.8
Bio Capabilities Index (SecureBio)Indexes153161+5.8
AA-PersonalityBench Warmth indexChat & Writing60.054.3−5.8
LMArena Agent Arena Steering BurdenAgents-1.310.1+5.7
Medical Long Context Reasoning (MLCR-AA)Health & Medicine21.716.1−5.6
HPCT (SecureBio)Science62.467.8+5.4
APEX-1 (Mercor, mean score)Professional Work68.863.6−5.2
LMArena Agent Arena SteerabilityAgents4.314.5+5.1
Harvey LAB: In-House Contracting (held-out)Legal6.31.3−5.0
CharXiv Extended (Mercor held-out)Multimodal78.983.8+4.9
Artificial Analysis Progress Index v1Indexes320334+4.8
Gray Swan IPI Attack Success (k=15)Safety5.510.1−4.6
DeepSWE v1.1Code71.066.6−4.4
τ³-Banking (Fireworks, banking_knowledge)Agents51.447.0−4.4
Epoch Capabilities Index (ECI)Indexes159164+4.3
ACCoRDGeneral QA78.282.4+4.2
APEX-Agents 1.1Professional Work72.968.7−4.2
Humanity's Last Exam (Text-Only)General QA42.146.3+4.2
SWE-bench MultilingualCode85.881.7−4.1
VCT v2 (SecureBio)Science45.249.3+4.1
CadQueryEval (Corrected Geometry)Code96.0100.0+4.0
ChessBench AccuracyGames81.177.4−4.0
LMArena Agent Arena Bash RecoveryAgents-0.77.0+3.9
Financial Coaching BenchFinance & Accounting66.362.6−3.7
BioSecBench-Surveillance V1 Clean RerunsScience44.147.7+3.6
Harvey LAB-AA v1.1 (hallucination-gated)Legal0.03.6+3.6
SWE-TogetherCode64.260.6−3.6
AutomationBench 1.0.6 (Zapier)Agents29.733.2+3.5
SWE Atlas - Codebase QnACode40.644.1+3.5
MedXpertQA MM (200-question set)Health & Medicine84.781.3−3.4
WCB (SecureBio)Science57.160.5+3.4
AA-Briefcase v1.1 EloAgents12041266+3.4
BioSecBench-Function V1 Clean RerunsScience36.039.3+3.3
AIME 2025Math96.7100.0+3.3
FACTS Benchmark SuiteGeneral QA69.472.5+3.1
ITBench-AAAgents52.549.4−3.1
ARC-AGI-1 Semi-PrivateGeneral QA98.595.5−3.0
SpatialBench-Long V1 Clean RerunsScience28.631.6+3.0
LAB-Bench ProtocolQA (SecureBio)Science84.081.1−2.9
Roboflow Vision Evals (October 2026, five tasks)Multimodal79.876.9−2.9
DAYJOB: HealthcareHealth & Medicine0.83.6+2.8
Roboflow Vision Evals (September 2026 protocol)Multimodal85.182.3−2.8
DistressBenchSafety70.768.0−2.8
Vals Index v2.1Indexes54.957.5+2.6
LMArena Agent Arena Praise vs ComplaintAgents4.910.0+2.5
MedScribeHealth & Medicine84.582.0−2.5
AURORA Multilingual Terminal-BenchCode67.765.3−2.5
ComplexConstraintsChat & Writing48.450.8+2.4
GDP.pdf (Artificial Analysis, All-pass)Professional Work22.825.2+2.4
Creative Writing v3 Rubric ScoreChat & Writing15.616.1+2.3
Terminal-Bench 2.1Code87.685.4−2.2
HealthBench ProfessionalHealth & Medicine59.061.2+2.2
ForecastBench Preliminary Dataset Brier IndexForecasting60.562.6+2.1
CVDP (517-problem set)Code50.752.7+2.0
Terminal-Bench 2.1 Extended (Mercor held-out)Code44.442.4−2.0
Chess Puzzles (task version 2.0)Games58.056.0−2.0
MMMU ProMultimodal84.582.5−2.0
Qodo AI Code Review Benchmark (125 PRs)Code47.048.9+1.9
MMLU ProGeneral QA90.788.9−1.8
GDPval-AA v2.1Professional Work14421402−1.8
Blueprint-Bench 2Multimodal0.40.4−1.7
BioSimBench 1.0Science15.313.6−1.7
GDPval Extended (Mercor held-out rubric score)Professional Work60.458.7−1.7
DTBenchGeneral QA95.797.3+1.6
LMArena Agent Arena Confirmed SuccessAgents1.34.5+1.6
Bedside BenchHealth & Medicine86.588.1+1.6
LAB-Bench CloningScenarios (SecureBio)Science100.098.5−1.5
MacroscopeBenchCode70.769.2−1.5
SciCode (main problems)Science14.212.7−1.5
FrontierMath ErdősMath0.01.5+1.5
Epiq LAER CorporateBenchProfessional Work77.679.0+1.4
NoCoT-Bench NCKI (kspine v3)General QA121119−1.3
AA-Omniscience IndexIndexes29.627.1−1.2
GDP.xlsxProfessional Work18.317.1−1.2
OTIS Mock AIME 2024-2025Math98.9100.0+1.1
MedCodeHealth & Medicine48.147.1−1.1
SciCode 1.0.1 (Artificial Analysis)Code56.657.6+1.0
Toloka Arena (v3)Agents59.058.0−1.0
Eval Connections One-Shot with TrapsGames88.089.0+1.0
ProgramBenchCode1.02.0+1.0
Design Arena Overall Code CategoriesCode12991293−1.0
APEX-Accounting 1.1Finance & Accounting58.859.7+0.9
AutomationBench-AA 1.0.6Agents60.961.7+0.8
ObviousBenchGeneral QA100.099.3−0.7
Excel Modeling BenchmarkFinance & Accounting72.271.5−0.7
EpiBench V1 Clean RerunsScience21.422.0+0.6
WMDP-Bio (SecureBio)Science89.490.0+0.6
LMArena Agent Arena Bash ReliabilityAgents9.910.9+0.5
ARC-AGI-2General QA89.289.6+0.4
CharXiv (descriptive and reasoning combined)Multimodal95.495.0−0.4
AA-LCR v1.1Search84.083.7−0.3
Vals Index v2Indexes62.362.6+0.3
LAB-Bench LitQA2 (SecureBio)Science51.551.8+0.3
SWE-bench VerifiedCode80.680.3−0.3
LMArena Agent Arena Tool HallucinationAgents0.41.0+0.3
SUPERChemScience72.672.8+0.2
LLM Chess EloGames154615450.0
GDPval (Mercor 213-task rubric score)Professional Work83.583.50.0

Gemini 3.8 FlashvsGPT-6 Solnormalised score marginsData Nov 15, 2026Data: Artificial Analysis, Vals AI, Mercor and 53 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…