Atlas

Compare Models

Gemini 3.8 Flash vs GPT-5.6 Sol

GPT-5.6 Sol leads by 2.2 capability points (≈95% interval −0.0 to +4.4): statistically tied · P(higher) ≈ 97%

Wins 149 of 259 shared benchmarks (103 lost, 7 tied)

Capability vs. relative cost
Gemini 3.8 FlashGPT-5.6 SolClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Gemini 3.8 FlashvsGPT-5.6 Solcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins103 – 149 · 259 shared, 7 tied

Comparing matching setups only (112 benchmarks excluded) · suites counted once
← Gemini 3.8 FlashGPT-5.6 Sol →

Gemini 3.8 FlashvsGPT-5.6 Solwins by categoryData Nov 15, 2026Data: Vals AI, Artificial Analysis, benchmarks.bio and 71 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-5.6 Sol minus Gemini 3.8 Flash.

FrontierMath Tier 4 (v2) - PrivateMath22.082.9+61.0
Olam Social Poker EloGames14791534+59.8
Short Form Bench v1.0Games19402410+46.6
BrokenArXiv 06/2026Math20.867.3+46.5
Concept Synth INDUCTION Challenge64 Round 1General QA14.157.8+43.8
MASKSafety58.695.4+36.8
ProofBench v1.1Math48.083.0+35.0
IOI (Vals v2)Code56.991.2+34.2
Concept Synth INDUCTION Challenge100 Round 1General QA9.043.0+34.0
CyberBench v1.1 OverallCybersecurity43.876.3+32.6
Design Creativity BenchCode60.651.3−29.5
Vending-Bench 2Games50949619+27.4
HieroglyphBenchMultimodal54.627.6−27.0
Hedge-BenchFinance & Accounting4.731.5+26.8
LLM Creative Story WritingChat & Writing0.12.5+26.2
ScBench V1 Clean RerunsScience34.059.8+25.8
Furniture Assembly BenchmarkMultimodal31.756.7+25.0
TaxCalcBench TY25Finance & Accounting6.030.0+24.0
TextQuests Average Cumulative HarmSafety24.431.4−23.7
RiemannBenchMath51.274.4+23.2
Shroombench (mushroom identification, top-1)Multimodal65.042.0−23.0
Gray Swan IPI Attack Success (k=15)Safety5.527.0−21.5
Antidote: Everyday EditionSafety10871027−21.4
ArXivMath 06/2026Math67.488.5+21.2
FrontierMath Tiers 1-3 (v2) - PrivateMath68.489.1+20.7
HardGeoBench <=100miMultimodal60.040.0−20.0
HardGeoBench <=10miMultimodal46.326.3−20.0
HardGeoBench Country AccuracyMultimodal67.547.5−20.0
SM BenchSafety80.461.1−19.3
LLM Debate BenchmarkChat & Writing15141645+18.6
ChessBench EloGames15271218−18.4
Terminal-Bench 4.0Code19.737.9+18.2
SAGEMath35.152.6+17.5
HardGeoBench Avg DistanceMultimodal8431381−16.6
Code MigrationCode36.552.9+16.4
Surface Evolver BenchAgents76.993.1+16.3
SWE-bench VerifiedCode80.096.2+16.2
GameBench 2Games64.080.1+16.1
MaxxBenchCode53.469.1+15.7
MindCube TinyMultimodal92.276.8−15.4
MacroscopeBenchCode51.366.1+14.8
Finance Agent v2 - Market AnalysisFinance & Accounting78.564.2−14.3
HPCT (SecureBio)Science62.476.3+13.9
GeneBench-Pro Public Release SubsetScience11.024.0+13.0
SWE Atlas - Codebase QnACode40.653.5+12.9
HANDBOOK.mdProfessional Work10.823.5+12.7
FrontierSWE V2Code19.632.2+12.6
EnterpriseBench: CoreCraftProfessional Work58.546.2−12.3
MBCT (SecureBio)Science48.961.1+12.2
BioTIER RefusalSafety83.971.9−12.0
CritPt v2 (agentic, avg@5)Science65.177.1+12.0
Finance Agent v2 - Financial ModelingFinance & Accounting32.620.7−11.9
TxBench Antibody Discovery V1 Clean RerunsScience43.331.4−11.9
Terminal-Bench-Science 0.1Science10.522.4+11.9
BioMysteryBench - Human DifficultScience56.544.7−11.8
APEX-SWECode36.448.0+11.6
Humanity's Sixth SenseMultimodal41.630.0−11.6
Creative Writing v3 EloChat & Writing17441974+11.6
Machiavelli Deception (CAIS)Safety86.582.7+11.5
Terminal-Bench Science 0.1 (Vals)Science8.620.0+11.4
Bio Capabilities Index (SecureBio)Indexes153169+11.2
HPCT Refusal (CAIS)Safety13.024.0+11.0
Mystery Game PuzzlesGames47.058.0+11.0
Agent Red Teaming k=100 (CAIS)Safety29.840.3−10.5
RareDiseaseBench V1 (35 evaluations)Science2.913.3+10.5
CritPt (avg@5)Science18.328.6+10.3
WellnessBenchSafety47.057.0+10.0
Hemingway-benchChat & Writing10931057−10.0
Finance Agent v2 - PrecedentsFinance & Accounting36.126.2−9.9
ScBench-Long V1 Clean RerunsScience19.929.6+9.7
BioMysteryBench - Human SolvableScience88.879.5−9.3
AA-Briefcase EloAgents12021361+9.3
BrowseComp Extended (Mercor held-out)Search41.532.3−9.2
SpeciEval (Refusal-Aware Retries, Historical)Safety89.999.0+9.2
SpeciEval (Pooled English Runs)Safety89.999.0+9.2
HiL-Bench (Human-in-Loop Benchmark)Agents41.532.3−9.1
Legal Research BenchLegal38.948.1+9.1
GBENCH One-shot Coding GScoreGames58.667.7+9.0
CrossSearch (pass^3)Search29.038.0+9.0
RedlineBenchLegal41.350.3+9.0
NoCoT-Bench NCRI 15.2General QA126109−9.0
SpatialBench-Long V1 Clean RerunsScience28.637.5+8.9
LiveBench 2.0Agents67.376.3+8.9
BioMysteryBench (Vals Terminus 2)Science62.271.1+8.9
SpatialBench V1 Clean RerunsScience59.468.1+8.7
Financial Coaching BenchFinance & Accounting66.374.9+8.6
OpenDS-WorldSearch36.528.0−8.5
ERQAMultimodal79.671.2−8.4
Arena Alignment IndexSafety75.884.2+8.4
Finance Agent v2 - All PassFinance & Accounting49.741.4−8.3
LMArena Agent Arena Net ImprovementAgents3.86.3+8.3
ARC-AGI-3 (Semi-Private)General QA10.42.1−8.2
ALE-BenchCode21822410+8.1
CadQueryEval (Corrected Geometry)Code96.088.0−8.0
Veracode GenAI Code Security (Fall 2026)Cybersecurity62.070.0+8.0
SpatialBench V0 VerifiedScience67.074.5+7.5
BrowseComp (Mercor 130-question subset)Search83.190.6+7.5
GDP.pdfProfessional Work23.230.7+7.5
Harvey's Legal Agent BenchmarkLegal10.02.5−7.5
Roboflow Vision Evals (October 2026, five tasks)Multimodal79.872.4−7.4
VCT (SecureBio)Science46.954.3+7.4
WCB (SecureBio)Science57.164.4+7.3
FrontierCode 1.1 Extended - ScoreCode53.560.6+7.2
Realm Financial ReasoningFinance & Accounting51.544.4−7.1
Finance Agent v2 - Earnings AnalysisFinance & Accounting81.474.4−7.0
Terminal-Bench 2.1 Extended (Mercor held-out)Code44.437.4−7.0
LiveCodeBench (Vals AI public dataset)Code89.582.6−6.9
Finance Agent v2 - ComparablesFinance & Accounting52.045.3−6.7
Continual Learning Exam (Trial 1)Agents21.227.8+6.7
TxBench Preclinical Pharmacology V1 Clean RerunsScience54.761.3+6.7
SWE-TogetherCode64.257.8−6.4
Roboflow Vision Evals (September 2026 protocol)Multimodal85.179.0−6.1
Artificial Analysis Intelligence Index v4.3.2Indexes40.947.0+6.0
Artificial Analysis Intelligence Index v4.3Indexes41.247.1+5.9
P2NPCBench (NP-complete, pass@1)Code62.668.4+5.8
Vals Science IndexIndexes35.841.5+5.7
Finance Agent v2 - AdjustmentsFinance & Accounting54.849.2−5.6
LMCAGeneral QA52.958.5+5.6
ChessBench CoherenceGames94.689.1−5.5
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine13.318.7+5.3
RT-SAFE Collisions per EpisodeSafety27.725.6+5.3
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting80.975.6−5.2
BuseyBench SVGMultimodal6.66.1−5.2
SciUniverse Level 1Science14.69.4−5.2
Finance Agent v2 - Disclosure AnalysisFinance & Accounting68.563.3−5.2
ChessBench AccuracyGames81.176.4−5.0
Blueprint-Bench 2Multimodal0.40.3−5.0
Eval Connections One-Shot with TrapsGames88.083.0−5.0
GDP.pdf - all-pass rate reported by GoogleProfessional Work35.040.0+5.0
Time Horizon Index: KSPAgents18.823.8+5.0
Longform Creative WritingChat & Writing76.881.7+4.9
VCT-Refusal (CAIS)Safety6.411.3+4.9
CVDP (517-problem set)Code50.755.4+4.7
MetagenomicsBench V1Science26.731.3+4.7
MetagenomicsBench V0Science26.731.3+4.6
Bedside BenchHealth & Medicine86.591.1+4.6
PCT Political Inconsistency (CAIS)Safety46.341.7+4.6
Medical Long Context Reasoning (MLCR-AA)Health & Medicine21.726.1+4.4
DeepSWE v1.1Code73.869.4−4.4
EpiBench V1 Clean RerunsScience21.425.8+4.4
LMArena Agent Arena Net Improvement (Steering Burden)Agents2.34.6+4.4
Harvey LAB: In-House Contracting (held-out)Legal6.32.0−4.3
VCT v2 (SecureBio)Science45.249.4+4.2
Conceptual Reasoning IndexIndexes65.768.7+4.2
MedCodeHealth & Medicine48.144.0−4.2
ChartographyMultimodal40.945.0+4.1
WeirdML (v2)Science84.888.8+4.0
VariantBench V1 Clean RerunsScience31.435.3+4.0
Artificial Analysis Progress Index v1Indexes320331+3.9
TxBench-Antibody-Development V1 (60 evaluations)Science15.018.9+3.9
SkillsBench with SkillsAgents58.054.1−3.9
AA-Omniscience IndexIndexes29.622.0−3.8
ITBench-AAAgents52.556.2+3.7
Extended NYT Connections 940 (Quadratic v1)Games97.493.8−3.6
Epoch Capabilities Index (ECI)Indexes159163+3.5
ObviousBenchGeneral QA100.096.5−3.5
LMArena Agent Arena Steering BurdenAgents-1.35.5+3.4
ARC-AGI-2General QA89.292.5+3.3
AURORA Multilingual Terminal-BenchCode67.764.5−3.2
FrontierFinance v3.1Finance & Accounting50.046.8−3.2
MysteryMechanismScience36.533.3−3.2
BioSecBench-Surveillance V1 Clean RerunsScience44.147.3+3.1
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting68.164.9−3.1
Vals Index v2.1Indexes54.958.0+3.1
Humanity's Last Exam DiamondGeneral QA34.331.2−3.1
GDPval-AA v2.1Professional Work14421511+3.1
AutoResearchExam (hidden-test AUARC)Agents0.50.5+3.0
GDPval-AA v2Professional Work14641524+3.0
MMLU Pro - LawGeneral QA82.079.1−2.9
VIALSMultimodal29.426.5−2.9
Creative Writing v3 Rubric ScoreChat & Writing15.616.2+2.9
DistressBenchSafety70.773.6+2.8
DAYJOB: FinanceFinance & Accounting2.85.5+2.7
SpatialViz-BenchMultimodal78.180.7+2.6
CRM-Agent-BenchAgents83.681.3−2.3
SkillsBench without SkillsAgents41.043.3+2.3
MMLU Pro - PsychologyGeneral QA91.589.2−2.3
ACCoRDGeneral QA78.275.9−2.2
LMArena Text Arena EloChat & Writing14971485−2.2
ComplexConstraintsChat & Writing48.450.5+2.1
APEX-Agents 1.1 Corporate Law (68-task image-free slice)Legal82.780.6−2.1
LAB-Bench ProtocolQA (SecureBio)Science84.081.9−2.1
TextQuests Game ProgressGames54.152.1−2.0
MMLU Pro - PhysicsGeneral QA94.892.8−2.0
SciCode (main problems)Science14.216.2+2.0
Toloka Arena (v3)Agents59.061.0+2.0
MMLU Pro - Computer ScienceGeneral QA92.790.7−2.0
MortgageTax (Vals all 1,258 documents)Finance & Accounting65.367.3+1.9
TxBench Oligo Discovery V1 Clean RerunsScience33.135.0+1.9
MMMU ProMultimodal84.582.7−1.8
Vibe Code Bench v1.1Code78.780.5+1.8
HealthBenchHealth & Medicine57.155.3−1.8
MMLU Pro - HistoryGeneral QA83.781.9−1.8
PG-LLM-50Science0.40.4+1.8
Document ArenaProfessional Work14821480−1.7
SUPERChemScience72.671.0−1.6
BioSecBench-RespiratoryVirus-Surveillance V1 (110 evaluations)Science7.35.8−1.5
Vals Index v2Indexes62.363.7+1.5
𝜏³-BankingAgents45.844.3−1.4
IntPhys 2Multimodal62.563.9+1.4
MMLU Pro - EconomicsGeneral QA92.190.6−1.4
TissueMicroscopyBench V1 (49 evaluations)Science3.42.0−1.4
Vision Arena OverallMultimodal12931288−1.3
LAB-Bench LitQA2 (SecureBio)Science51.552.8+1.3
AA-AnalystAgent (pass^5)Professional Work46.347.5+1.3
Public Benefits Bench v1.1Professional Work65.366.5+1.2
LAB-Bench CloningScenarios (SecureBio)Science100.098.8−1.2
Tax Agent BenchFinance & Accounting66.868.0+1.2
SciCode 1.0.1 (Artificial Analysis)Code56.657.8+1.2
LMArena Agent Arena SteerabilityAgents4.36.5+1.1
MMLU Pro (Vals 5-Shot CoT)General QA90.289.1−1.1
OTIS Mock AIME 2024-2025Math98.9100.0+1.1
Terminal-Bench 4.0 (Artificial Analysis)Code19.720.7+1.0
Chess Puzzles (task version 2.0)Games58.059.0+1.0
ARC-AGI-1 Semi-PrivateGeneral QA98.597.5−1.0
WordleBenchGames98.097.0−1.0
RCM-Bench-1Health & Medicine90.291.2+1.0
MMLU Pro - EngineeringGeneral QA88.887.8−0.9
AutomationBench 1.0.6 (Zapier)Agents29.728.8−0.9
ForecastBench Preliminary Dataset Brier IndexForecasting60.561.4+0.9
MMLU Pro - MathMath96.295.3−0.9
AutomationBench-AA 1.0.6Agents60.960.1−0.8
DAYJOB: HealthcareHealth & Medicine0.81.6+0.8
LMArena Agent Arena Bash ReliabilityAgents9.911.4+0.8
MedScribeHealth & Medicine84.585.2+0.7
LMArena Agent Arena Bash RecoveryAgents-0.70.7+0.7
LMArena Agent Arena Praise vs ComplaintAgents4.96.3+0.7
Humanity's Last ExamGeneral QA46.245.5−0.7
SWE-Bench Pro V2 — FullCode94.995.5+0.6
MMLU Pro - ChemistryGeneral QA92.592.0−0.5
Political Science LLM BenchmarkProfessional Work68.769.2+0.5
MMLU Pro - BusinessGeneral QA93.893.3−0.5
ProgramBenchCode1.01.5+0.5
LMArena Agent Arena Confirmed SuccessAgents1.32.3+0.5
BioSecBench-Function V1 Clean RerunsScience36.035.5−0.4
Prophet Arena Arena Score (Fixed Context)Forecasting85.384.8−0.4
MMLU Pro - BiologyGeneral QA95.094.6−0.4
Terminal-Bench 2.1Code87.687.3−0.4
MMLU Pro - HealthGeneral QA84.584.1−0.4
SEA-HELMGeneral QA77.377.0−0.4
ATLAS-Finance v1.1Finance & Accounting3.74.0+0.3
TaxEval v2Finance & Accounting74.474.8+0.3
WMDP-Bio (SecureBio)Science89.489.7+0.3
LMArena Agent Arena Tool HallucinationAgents0.41.0+0.3
MMLU Pro - PhilosophyGeneral QA88.288.4+0.2
APEX-AccountingFinance & Accounting51.751.5−0.2
LLM Chess EloGames15461550+0.1
Excel Modeling BenchmarkFinance & Accounting72.272.3+0.1
MMLU Pro - OtherGeneral QA87.687.7+0.1
LegalBenchLegal87.087.00.0
NoCoT-Bench NCKI (kspine v3)General QA1211210.0
DTBenchGeneral QA95.795.70.0
AA-LCR v1.1Search84.084.00.0
ATLAS-Finance v1.0Finance & Accounting4.74.70.0
BabaIsBench Stage 0: The IntroGames100.0100.00.0
FrontierMath ErdősMath0.00.00.0
GPQA DiamondGeneral QA94.994.90.0
NeuroscienceBench V1 (91 evaluations)Science15.815.80.0
ORCA-bench Hard RCA (Fireworks 324-task subset)Agents29.529.50.0

Gemini 3.8 FlashvsGPT-5.6 Solnormalised score marginsData Nov 15, 2026Data: Vals AI, Artificial Analysis, benchmarks.bio and 71 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…