Atlas

Compare Models

Gemini 3.8 Flash vs GPT-6.1 Sol

GPT-6.1 Sol leads by 5.6 capability points (≈95% interval +3.2 to +8.0): a clear lead over Gemini 3.8 Flash · P(higher) ≈ 100%

Wins 208 of 270 shared benchmarks (57 lost, 5 tied)

Capability vs. relative cost
Gemini 3.8 FlashGPT-6.1 SolClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Gemini 3.8 FlashvsGPT-6.1 Solcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins57 – 208 · 270 shared, 5 tied

Comparing matching setups only (84 benchmarks excluded) · suites counted once
← Gemini 3.8 FlashGPT-6.1 Sol →

Gemini 3.8 FlashvsGPT-6.1 Solwins by categoryData Nov 15, 2026Data: Vals AI, Artificial Analysis, Mercor and 58 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-6.1 Sol minus Gemini 3.8 Flash.

Short Form Bench v1.0Games19402949+100.0
Concept Synth INDUCTION Challenge64 Round 1General QA14.195.3+81.3
Concept Synth INDUCTION Challenge100 Round 1General QA9.090.0+81.0
BrokenArXiv 06/2026Math20.8100.0+79.2
FrontierMath Tier 4 (v2) - PrivateMath22.0100.0+78.0
BrokenArXiv 05/2026Math19.090.0+71.0
Olam Social Poker EloGames14791543+69.6
AA-PersonalityBench Transparency indexChat & Writing23.892.4+69.2
ViBench 2.0Code0.064.7+64.7
SAFE-Teen 106 worst branch: Health Advice (no prompt)Safety69.08.8+60.2
ARC-AGI-3 (Semi-Private)General QA35.095.0+60.0
AA-PersonalityBench Slop indexChat & Writing71.414.3−57.7
Clipping Bench v1.0Games5965707+55.0
SAFE-Teen 106: Health Advice (no prompt)Safety59.64.7+54.9
SAFE-Teen Health Advice (no prompt)Safety59.14.2+54.8
AA-PersonalityBench Informativeness indexChat & Writing81.027.7−53.8
ProofBench v1.1Math48.099.0+51.0
Furniture Assembly BenchmarkMultimodal31.780.0+48.3
Terminal-Bench Science 0.1 (Vals)Science8.652.9+44.3
SAFE-Teen 106 worst branch: Relationships (no prompt)Safety62.619.1+43.5
SAFE-Teen 106 worst branch: Urgent Safety (no prompt)Safety56.415.4+41.1
SAFE-Teen 106 worst branch: Health Advice (under-18 prompt)Safety48.17.7+40.4
IOI (Vals v2)Code56.996.9+39.9
MASKSafety58.698.3+39.7
Code Migration v1.1Code38.677.3+38.7
Terminal-Bench 4.0Code19.758.2+38.5
Code2Video-AA EloCode8991369+38.2
ArXivMath 05/2026Math58.395.0+36.7
Terminal-Bench-Science 0.1Science10.046.2+36.2
SAFE-Teen 106: Health Advice (under-18 prompt)Safety37.53.3+34.2
SAFE-Teen Health Advice (under-18 prompt)Safety36.12.0+34.1
Mystery Game PuzzlesGames47.080.0+33.0
TaxCalcBench TY25Finance & Accounting6.038.0+32.0
Terminal-Bench 4.0 (Artificial Analysis)Code19.751.5+31.8
SAFE-Teen Relationships (no prompt)Safety41.710.0+31.7
AA-PersonalityBench Conviction indexChat & Writing84.854.3−30.8
CrossSearch (pass^3)Search29.059.0+30.0
SAFE-Teen 106: Relationships (no prompt)Safety40.110.2+29.9
Humanity's Last Exam DiamondGeneral QA29.158.8+29.7
ScBench V1 Clean RerunsScience34.063.6+29.6
HPCT Refusal (CAIS)Safety13.042.0+29.0
AA-PersonalityBench Expressiveness indexChat & Writing79.150.5−28.8
ALE-BenchCode21822990+28.6
Code MigrationCode36.565.1+28.6
SAFE-Teen 106: Urgent Safety (no prompt)Safety31.53.4+28.1
GameBench 2Games64.092.0+28.0
SAFE-Teen Urgent Safety (no prompt)Safety31.03.5+27.5
ArXivMath 06/2026Math67.494.4+27.1
BrowseComp Extended (Mercor held-out)Search41.567.7+26.2
FrontierMath Tiers 1-3 (v2) - PrivateMath68.493.7+25.3
MMMU-Pro Extended (Mercor held-out)Multimodal36.461.3+24.9
RareDiseaseBench V1 (35 evaluations)Science2.927.6+24.8
Artificial Analysis Intelligence Index v5.0Indexes30.554.5+24.0
Vals Science IndexIndexes35.859.6+23.9
Artificial Analysis Intelligence Index v5.0.1Indexes30.854.6+23.8
sudo L7 (October 2026 rescoring)Code10.633.9+23.3
Machiavelli Deception (CAIS)Safety86.578.8+23.3
HieroglyphBenchMultimodal54.631.4−23.2
RiemannBench (November 2026 rescoring)Math56.879.2+22.4
EnigmaEvalGeneral QA37.259.5+22.3
Decision-Bench v1.1Professional Work26.949.1+22.2
SAFE-Teen 106 worst branch: Urgent Safety (under-18 prompt)Safety34.812.7+22.1
GeneBench-Pro Public Release SubsetScience11.033.0+22.0
AA-3D v1.0 EloCode9741121+21.3
AA-PersonalityBench Imagination indexChat & Writing41.061.9+21.2
HoneyBench v0.2 (hack rate)Safety80.559.5+21.0
HiL-Bench (Human-in-Loop Benchmark)Agents41.562.3+20.9
GBENCH One-shot Coding GScoreGames58.679.0+20.4
VIALSMultimodal29.449.1+19.7
DAYJOB: AccountingFinance & Accounting6.025.2+19.2
LMArena Agent Arena Net Improvement (Steering Burden)Agents2.312.5+19.2
TissueMicroscopyBench V1 (49 evaluations)Science3.422.4+19.0
MetagenomicsBench V1Science26.745.7+19.0
WellnessBenchSafety47.066.0+19.0
Hermes IndexIndexes38.157.1+19.0
LiveBench 2.0Agents67.385.4+18.1
FrontierCode 2.0 (Artificial Analysis)Code18.736.7+18.0
TxBench Oligo Discovery V1 (113 evaluations)Science35.152.5+17.4
BioMysteryBench (Vals Terminus 2)Science62.279.6+17.4
TextQuests Game ProgressGames54.171.4+17.3
CritPt v2 (agentic, avg@5)Science65.182.0+16.9
TextQuests Average Cumulative HarmSafety24.429.3−16.7
DAYJOB: FinanceFinance & Accounting2.818.0+15.2
SheetBench-50Finance & Accounting45.660.5+14.9
VariantBench V1 Clean RerunsScience31.446.0+14.7
AA-Briefcase v1.1 EloAgents12041469+14.5
Finance Agent v2 - Market AnalysisFinance & Accounting78.564.5−14.0
CritPt (avg@5)Science18.331.7+13.4
BuseyBench SVGMultimodal6.68.0+13.2
SAFE-Teen Urgent Safety (under-18 prompt)Safety15.22.1+13.1
SAFE-Teen 106: Urgent Safety (under-18 prompt)Safety14.92.0+12.9
TxBench-Antibody-Development V1 (60 evaluations)Science15.027.8+12.8
NoCoT-Bench NCRI 15.2General QA126149+12.8
SnorkelManufacturingProfessional Work7.119.6+12.5
MKA (Artificial Analysis)Professional Work35.347.7+12.4
BrowseComp (Mercor 130-question subset)Search83.195.4+12.3
Medical Long Context Reasoning (MLCR-AA)Health & Medicine21.733.9+12.2
SpatialViz-BenchMultimodal78.190.3+12.2
Arena Alignment IndexSafety75.887.9+12.1
GDP.pdfProfessional Work23.235.0+11.8
Hermes BenchAgents62.273.9+11.7
Web Clone BenchCode1.012.6+11.5
SAGEMath35.146.5+11.5
SM BenchSafety80.469.0−11.5
SpatialBench-Long V1 Clean RerunsScience28.639.9+11.3
LMArena Agent Arena Praise vs ComplaintAgents4.927.5+11.3
Agent Red Teaming k=100 (CAIS)Safety29.818.8+11.0
Artificial Analysis Intelligence Index v4.3.2Indexes40.951.8+10.9
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine13.324.0+10.7
APEX-SWECode36.446.9+10.5
Finance Agent v2 - PrecedentsFinance & Accounting36.125.8−10.4
Vibe Code Bench v1.1Code78.788.9+10.3
FrontierCode 1.1 Main (Artificial Analysis)Code36.446.6+10.2
Snorkel Agentic Coding 2.0Code32.642.8+10.2
HealthBench (length-adjusted, Artificial Analysis)Health & Medicine48.458.5+10.1
Finance Agent v2 - AdjustmentsFinance & Accounting54.844.7−10.1
Finance Agent v2 - Disclosure AnalysisFinance & Accounting68.558.4−10.1
BioSecBench-Function V1 Clean RerunsScience36.046.0+10.1
Chess Puzzles (task version 2.0)Games58.068.0+10.0
Design Creativity BenchCode60.663.7+9.9
Finance Agent v2 - Earnings AnalysisFinance & Accounting81.471.5−9.9
MysteryMechanismScience36.546.4+9.9
Artificial Analysis Progress Index v1Indexes320348+9.9
LMCAGeneral QA52.962.8+9.9
IntPhys 2Multimodal62.572.3+9.8
BioSecBench-Refusal V1 Clean RerunsSafety47.437.6−9.8
Finance Agent v2 - All PassFinance & Accounting49.739.9−9.8
VCT-Refusal (CAIS)Safety6.416.1+9.7
Conceptual Reasoning IndexIndexes65.772.7+9.6
Finance Agent v2 - Financial ModelingFinance & Accounting32.623.1−9.6
SAFE-Teen 106 worst branch: Relationships (under-18 prompt)Safety14.04.5+9.5
Design Arena Overall Code CategoriesCode12991356+9.5
Finance Agent v2 - ComparablesFinance & Accounting52.042.6−9.4
HealthBench ProfessionalHealth & Medicine59.068.3+9.3
PCT Political Inconsistency (CAIS)Safety46.337.3+9.0
Visa Agentic Commerce BenchmarkAgents53.362.2+8.9
SWE-bench Verified Extended (Mercor held-out)Code54.045.3−8.7
SWE Atlas v1.1 - OverallCode57.365.9+8.6
Epoch Capabilities Index (ECI)Indexes159169+8.3
EnterpriseBench: CoreCraftProfessional Work58.550.3−8.2
Vals Index v2.2Indexes56.063.9+8.0
SpatialBench V1 Clean RerunsScience59.467.3+7.8
SWE Atlas - Codebase QnACode40.648.4+7.8
AA-PersonalityBench Organization indexChat & Writing73.381.0+7.7
SpeciEval (Refusal-Aware Retries, Historical)Safety89.997.4+7.5
SpeciEval (Pooled English Runs)Safety89.997.4+7.5
Bedside BenchHealth & Medicine86.593.9+7.4
LMArena Agent Arena Steering BurdenAgents-1.313.4+7.3
ChessBench CoherenceGames94.687.5−7.1
Toloka Arena (v3)Agents59.066.0+7.0
EpiBench V1 Clean RerunsScience21.428.3+6.9
ComplexConstraints (November 2026 rescoring)Chat & Writing63.270.1+6.9
ScBench-Long V1 Clean RerunsScience19.926.7+6.8
DAYJOB: HealthcareHealth & Medicine0.87.6+6.8
τ³-Banking (Fireworks, banking_knowledge)Agents51.444.7−6.7
MissionReportProfessional Work54.361.0+6.7
SWE-bench VerifiedCode80.687.3+6.7
SnorkelRevOpsProfessional Work5.412.1+6.7
BioSecBench-Surveillance V1 Clean RerunsScience44.150.8+6.7
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting80.974.5−6.3
Vals Index v2.1Indexes54.961.2+6.3
NeuroscienceBench V1 (91 evaluations)Science15.822.0+6.2
CharXiv Extended (Mercor held-out)Multimodal78.985.1+6.2
NoCoT-Bench NCKI (kspine v3)General QA121130+6.2
AA-Omniscience IndexIndexes29.641.5+6.0
Public Benefits Bench v1.1Professional Work65.359.3−6.0
FACTS Benchmark SuiteGeneral QA69.475.3+5.9
SWE-TogetherCode64.258.3−5.9
SnorkelUnderwrite 2.0Finance & Accounting19.125.0+5.9
HLAB-Vals-1.1Legal12.16.3−5.8
τ³-Banking (Mercor run, bm25_grep retrieval)Agents48.843.0−5.8
AA-PersonalityBench Warmth indexChat & Writing60.065.7+5.8
AutomationBench-AA 1.0.6Agents60.966.6+5.7
APEX-Accounting 1.1Finance & Accounting58.864.3+5.5
Westworld Finance Diligence BenchFinance & Accounting49.755.0+5.3
DAXBench v2Code87.292.6+5.3
SUPERChemScience72.677.8+5.2
LMArena Agent Arena Confirmed SuccessAgents1.311.7+5.2
BioSecBench-RespiratoryVirus-Surveillance V1 (110 evaluations)Science7.312.4+5.1
Humanity's Last ExamGeneral QA46.251.3+5.1
TxBench Preclinical Pharmacology V1 Clean RerunsScience54.759.7+5.0
ARC-AGI-2General QA89.294.2+5.0
Humanity's Sixth SenseMultimodal41.646.6+5.0
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting68.163.2−4.9
ChessBenchGames73.368.7−4.7
Vals Finance IndexIndexes60.555.8−4.7
Epiq LAER CorporateBenchProfessional Work77.682.2+4.6
SnorkelLegalLegal25.320.7−4.6
Harvey's Legal Agent BenchmarkLegal10.05.4−4.6
CyberBench v1.1 OverallCybersecurity43.839.3−4.5
Tax Agent BenchFinance & Accounting66.862.3−4.5
FrontierMath ErdősMath0.04.4+4.4
Immigration BenchLegal63.467.8+4.4
CVDP (517-problem set)Code50.755.1+4.4
Snorkel Digital Work IndexIndexes16.921.2+4.3
DeepSWE v1.1Code71.075.2+4.2
SimpleQA Verified (Epoch anti-abstention)General QA69.773.9+4.2
Finance Agent v2.1Finance & Accounting65.861.8−3.9
MedXpertQA MM Extended (Mercor held-out)Health & Medicine60.864.7+3.9
Snorkel WorkplaceAgentsProfessional Work11.215.0+3.8
AA-AnalystAgent (pass^5)Professional Work46.350.0+3.8
GDP.xlsxProfessional Work18.322.0+3.7
Snorkel SWE-bench CLICode6.39.8+3.5
GDPval-AA v2.1Professional Work14421521+3.5
DistressBenchSafety70.774.2+3.4
AIME 2025Math96.7100.0+3.3
Professional Reasoning Benchmark - FinanceFinance & Accounting49.546.2−3.3
DAYJOB: TaxFinance & Accounting0.84.0+3.2
SAFE-Teen Relationships (under-18 prompt)Safety4.00.9+3.1
Gray Swan IPI Attack Success (k=15)Safety5.58.6−3.1
MindCube TinyMultimodal92.289.1−3.1
RCM-Bench-1Health & Medicine90.287.4−2.9
SAFE-Teen 106: Relationships (under-18 prompt)Safety3.70.9+2.9
MacroscopeBenchCode70.773.5+2.8
ACCoRDGeneral QA78.280.9+2.7
SWE-bench Multimodal (Vals v1, 480 Tasks)Code40.843.5+2.7
LMArena Text Arena EloChat & Writing14971482−2.7
LLM Chess EloGames15461612+2.7
LMArena Agent Arena Bash RecoveryAgents-0.74.5+2.6
Harvey LAB (Mercor run, undisclosed ~213-task cohort)Legal11.48.8−2.6
LMArena Agent Arena Bash ReliabilityAgents9.95.0−2.5
ScarfBench Smoke ScoreCode66.068.4+2.4
Financial Coaching BenchFinance & Accounting66.368.5+2.3
SAGE v1.1Math88.290.5+2.3
Terminal-Bench 2.1Code87.685.4−2.2
HANDBOOK.md Agents v1.1Professional Work26.528.6+2.1
ChessBench EloGames15271562+2.1
DualEntry Accounting AI BenchmarkFinance & Accounting83.281.2−2.0
ProgramBenchCode1.03.0+2.0
MedScribeHealth & Medicine84.586.5+2.0
Extended NYT Connections 940 (Quadratic v1)Games97.495.5−1.9
Roboflow Vision Evals (October 2026, five tasks)Multimodal79.881.7+1.9
Excel Modeling Benchmark v1.1Finance & Accounting72.971.1−1.8
Harvey LAB: In-House Contracting (held-out)Legal6.34.7−1.7
ERQAMultimodal79.678.2−1.4
Excel Modeling BenchmarkFinance & Accounting72.270.8−1.4
Terminal-Bench 2.1 Extended (Mercor held-out)Code44.445.8+1.4
SWE-Bench Pro V2 — FullCode94.996.1+1.3
Access to Justice BenchmarkLegal67.165.8−1.2
ForecastBench Preliminary Dataset Brier IndexForecasting60.561.7+1.2
SnorkelFinance 2.0Finance & Accounting5.46.6+1.2
OTIS Mock AIME 2024-2025Math98.9100.0+1.1
Vals Legal IndexIndexes69.868.7−1.1
Professional Reasoning Benchmark - LegalLegal48.947.9−1.0
MedXpertQA MM (200-question set)Health & Medicine84.785.7+1.0
SWE-bench MultilingualCode85.884.9−0.9
SciCode 1.0.1 (Artificial Analysis)Code56.655.8−0.8
DTBenchGeneral QA95.796.5+0.8
MedCodeHealth & Medicine48.148.8+0.7
TxBench Antibody Discovery V1 (100 evaluations)Science49.049.7+0.7
Ramp Accounting BenchFinance & Accounting45.746.4+0.6
CharXiv (descriptive and reasoning combined)Multimodal95.496.0+0.6
MMMU ProMultimodal84.585.1+0.6
SimpleBenchGeneral QA82.482.9+0.5
Legal Research BenchLegal38.938.5−0.5
Prophet Arena Arena Score (Fixed Context)Forecasting85.384.8−0.4
ChessBench AccuracyGames81.180.7−0.4
Roboflow Vision Evals (September 2026 protocol)Multimodal85.185.5+0.4
SciCode (main problems)Science14.214.6+0.4
APEX-Agents 1.1Professional Work72.973.2+0.3
MMLU ProGeneral QA90.791.0+0.3
LMArena Agent Arena Tool HallucinationAgents0.41.0+0.3
HealthBenchHealth & Medicine57.156.9−0.2
Vision Arena OverallMultimodal12931293−0.1
Prophet Arena Arena Score (Agentic, OpenCode)Forecasting86.086.00.0
AA-LCR v1.1Search84.084.00.0
ARC-AGI-1 Semi-PrivateGeneral QA98.598.50.0
BioSimBench 1.0Science15.315.30.0
Dual-Arm Sim (task completion)Agents0.00.00.0
ObviousBenchGeneral QA100.0100.00.0

Gemini 3.8 FlashvsGPT-6.1 Solnormalised score marginsData Nov 15, 2026Data: Vals AI, Artificial Analysis, Mercor and 58 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…