Atlas

Compare Models

Opus 4.8 vs Gemini 3.8 Flash

Gemini 3.8 Flash leads by 0.9 capability points (≈95% interval −1.1 to +3.0): statistically tied · P(higher) ≈ 82%

Wins 118 of 209 shared benchmarks (89 lost, 2 tied)

Capability vs. relative cost
Opus 4.8Gemini 3.8 FlashClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Opus 4.8vsGemini 3.8 Flashcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins89 – 118 · 209 shared, 2 tied

Comparing matching setups only (66 benchmarks excluded) · suites counted once
← Opus 4.8Gemini 3.8 Flash →

Opus 4.8vsGemini 3.8 Flashwins by categoryData Nov 15, 2026Data: Vals AI, Mercor, Artificial Analysis and 51 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Gemini 3.8 Flash minus Opus 4.8.

HPCT Refusal (CAIS)Safety85.013.0−72.0
LLM Chess EloGames4501546+45.0
Olam Social Poker EloGames15181479−42.4
HieroglyphBenchMultimodal14.054.6+40.6
MASKSafety94.458.6−35.8
VCT-Refusal (CAIS)Safety41.26.4−34.8
FrontierMath Tier 4 (v2) - PrivateMath56.122.0−34.1
ChessBench EloGames9611527+33.7
CadQueryEval (Corrected Geometry)Code68.096.0+28.0
MindCube TinyMultimodal64.992.2+27.3
Blueprint-Bench 2Multimodal0.10.4+24.1
Medical Long Context Reasoning (MLCR-AA)Health & Medicine45.621.7−23.9
ScBench V1 Clean RerunsScience57.134.0−23.1
BlenderWorldCode57.034.2−22.8
DeepSWE v1.1Code51.873.8+22.1
MetagenomicsBench V1Science48.326.7−21.7
MetagenomicsBench V0Science48.326.7−21.6
Roboflow Vision Evals (October 2026, five tasks)Multimodal58.379.8+21.5
BrokenArXiv 06/2026Math41.720.8−20.8
Antidote: Everyday EditionSafety10301087+20.4
ERQAMultimodal59.579.6+20.1
LVBenchMultimodal67.387.1+19.8
SAGEMath54.835.1−19.7
Humanity's Sixth SenseMultimodal23.541.6+18.1
SimpleBenchGeneral QA64.882.4+17.6
Agent Red Teaming k=100 (CAIS)Safety47.029.8+17.2
ARC-AGI-2General QA72.189.2+17.1
EnigmaEvalGeneral QA20.537.2+16.7
SimpleQA Verified (Epoch anti-abstention)General QA53.069.7+16.7
LLM Debate BenchmarkChat & Writing16311514−16.6
Roboflow Vision Evals (September 2026 protocol)Multimodal68.785.1+16.4
FACTS Benchmark SuiteGeneral QA53.569.4+15.8
AutomationBench-AA 1.0.6Agents45.660.9+15.3
Hemingway-benchChat & Writing10381093+15.2
ChessBench CoherenceGames79.794.6+14.9
BuseyBench SVGMultimodal5.26.6+14.2
BioMysteryBench - Human DifficultScience42.456.5+14.1
TaxCalcBench TY25Finance & Accounting20.06.0−14.0
TextQuests Game ProgressGames40.154.1+14.0
TxBench Antibody Discovery V1 Clean RerunsScience30.243.3+13.1
NoCoT-Bench NCRI 15.2General QA102126+13.0
SpatialViz-BenchMultimodal65.278.1+12.9
ComplexConstraintsChat & Writing35.648.4+12.8
AutomationBench 1.0.6 (Zapier)Agents16.929.7+12.8
HLE Extended (Mercor held-out)General QA49.436.8−12.6
SUPERChemScience60.372.6+12.3
MedXpertQA MM (200-question set)Health & Medicine72.584.7+12.2
Terminal-Bench 2.1 Extended (Mercor held-out)Code32.744.4+11.7
FrontierMath Tiers 1-3 (v2) - PrivateMath80.068.4−11.6
TextQuests Average Cumulative HarmSafety21.024.4−11.6
𝜏³-BankingAgents34.245.8+11.5
BrowseComp Extended (Mercor held-out)Search30.041.5+11.5
Finance Agent v2 - Market AnalysisFinance & Accounting67.278.5+11.3
HANDBOOK.mdProfessional Work21.910.8−11.1
VariantBench V1 Clean RerunsScience42.431.4−11.0
Mystery Game PuzzlesGames36.047.0+11.0
Furniture Assembly BenchmarkMultimodal42.531.7−10.8
BioTIER RefusalSafety94.783.9−10.8
Code MigrationCode47.336.5−10.7
Surface Evolver BenchAgents87.576.9−10.6
Finance Agent v2 - PrecedentsFinance & Accounting25.636.1+10.5
LMArena Agent Arena Net ImprovementAgents6.93.8−10.3
WCB (SecureBio)Science66.757.1−9.6
LiveBench 2.0Agents76.867.3−9.4
BrowseComp (Mercor 130-question subset)Search73.783.1+9.4
ChessBench AccuracyGames72.481.1+9.3
Finance Agent v2 - Earnings AnalysisFinance & Accounting72.181.4+9.3
SWE-bench Verified Extended (Mercor held-out)Code63.354.0−9.3
Finance Agent v2 - Financial ModelingFinance & Accounting23.332.6+9.3
Finance Agent v2 - General Quantitative AnalysisFinance & Accounting71.680.9+9.2
MedXpertQA MM Extended (Mercor held-out)Health & Medicine51.660.8+9.2
AA-Briefcase EloAgents13541202−8.9
ARC-AGI-3 (Semi-Private)General QA1.510.4+8.8
SWE-bench VerifiedCode88.680.0−8.6
APEX-Agents 1.1Professional Work64.472.9+8.5
Machiavelli Deception (CAIS)Safety83.786.5−8.4
Concept Synth INDUCTION Challenge64 Round 1General QA6.314.1+7.8
Design Arena Overall Code CategoriesCode12521299+7.8
CharXiv ReasoningMultimodal78.586.2+7.7
AA-LCR Extended (Mercor held-out)Search63.255.6−7.6
APEX-SWECode43.936.4−7.5
SM BenchSafety72.980.4+7.5
MBCT (SecureBio)Science56.448.9−7.5
PCT Political Inconsistency (CAIS)Safety53.846.3+7.5
IntPhys 2Multimodal55.162.5+7.4
Bio Capabilities Index (SecureBio)Indexes164153−7.4
Finance Agent v2 - ComparablesFinance & Accounting44.952.0+7.2
BioSecBench-Function V1 Clean RerunsScience43.136.0−7.1
LAB-Bench ProtocolQA (SecureBio)Science76.984.0+7.1
GeneBench-Pro Public Release SubsetScience18.011.0−7.0
Time Horizon Index: KSPAgents11.818.8+7.0
LLM Creative Story WritingChat & Writing0.80.1−6.9
ArXivMath 05/2026Math65.058.3−6.7
Document ArenaProfessional Work14751482+6.6
Finance Agent v2 - AdjustmentsFinance & Accounting48.454.8+6.4
SpatialBench V1 Clean RerunsScience65.859.4−6.4
Extended NYT Connections 940 (Quadratic v1)Games91.197.4+6.3
EnterpriseBench: CoreCraftProfessional Work52.358.5+6.2
Finance Agent v2 - All PassFinance & Accounting43.549.7+6.2
FrontierCode 1.1 Extended - ScoreCode59.653.5−6.1
HiL-Bench (Human-in-Loop Benchmark)Agents35.341.5+6.1
APEX-1 (Mercor, mean score)Professional Work62.768.8+6.1
Humanity's Last Exam DiamondGeneral QA23.129.1+6.0
ARC-AGI-1 Semi-PrivateGeneral QA92.598.5+6.0
Chess Puzzles (task version 2.0)Games52.058.0+6.0
FrontierCode 1.1 Extended - Pass RateCode65.559.6−5.9
Vals RSI Index v1.1 (historical five tasks)Agents23.217.6−5.7
Terminal-Bench 2.1Code82.087.6+5.6
SpeciEval (Pooled English Runs)Safety84.389.9+5.6
SpeciEval (Refusal-Aware Retries, Historical)Safety84.389.9+5.5
MedCodeHealth & Medicine53.248.1−5.1
MMMU-Pro Extended (Mercor held-out)Multimodal31.436.4+5.0
FrontierFinance v3.1Finance & Accounting45.050.0+5.0
Legal Research BenchLegal43.838.9−4.8
SpatialBench-Long V1 Clean RerunsScience33.328.6−4.8
GBENCH One-shot Coding GScoreGames54.058.6+4.6
LMCAGeneral QA57.552.9−4.6
MortgageTax (Vals all 1,258 documents)Finance & Accounting69.965.3−4.6
EpiBench V1 Clean RerunsScience25.821.4−4.4
LAB-Bench CloningScenarios (SecureBio)Science95.9100.0+4.1
Vibe Code Bench v1.1Code82.778.7−4.1
Vals RSI Index v1.1 (four tasks)Agents24.019.9−4.1
BabaIsBench Stage 0: The IntroGames96.0100.0+4.0
BrokenArXiv 05/2026Math23.019.0−4.0
RiemannBenchMath47.251.2+4.0
WordleBenchGames94.098.0+4.0
LMArena Agent Arena Bash RecoveryAgents7.2-0.7−4.0
SWE Atlas - Codebase QnACode44.440.6−3.8
APEX-AccountingFinance & Accounting48.051.7+3.7
Humanity's Last ExamGeneral QA49.846.2−3.6
Conceptual Reasoning IndexIndexes68.365.7−3.6
LMArena Code Arena WebDevCode15551583+3.6
Terminal-Bench 4.0Code23.219.7−3.5
LegalBenchLegal83.687.0+3.4
ScBench-Long V1 Clean RerunsScience16.519.9+3.4
GDPval-AA v2Professional Work15291464−3.2
GPQA DiamondGeneral QA91.994.9+3.0
TxBench Preclinical Pharmacology V1 Clean RerunsScience57.754.7−3.0
WMDP-Bio (SecureBio)Science92.389.4−2.9
Bug Hunt BenchCode14.317.1+2.9
Public Benefits Bench v1.1Professional Work68.165.3−2.8
ALE-BenchCode21022182+2.8
Excel Modeling BenchmarkFinance & Accounting69.472.2+2.8
LAB-Bench LitQA2 (SecureBio)Science48.751.5+2.8
BioSecBench-Surveillance V1 Clean RerunsScience46.944.1−2.8
Finance Agent v2 - Disclosure AnalysisFinance & Accounting65.768.5+2.8
LMArena Text Arena EloChat & Writing14821497+2.7
SWE-TogetherCode61.564.2+2.7
SciCode (main problems)Science16.914.2−2.7
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine16.013.3−2.7
LMArena Agent Arena SteerabilityAgents9.64.3−2.7
SkillsBench without SkillsAgents43.741.0−2.6
ArXivMath 06/2026Math70.067.4−2.6
Gray Swan IPI Attack Success (k=15)Safety8.05.5+2.5
Remote Labor Index (RLI)Professional Work8.35.8−2.5
MMMU-Pro (Vals 4-Option)Multimodal86.689.1+2.5
NoCoT-Bench NCKI (kspine v3)General QA117121+2.4
TxBench Oligo Discovery V1 Clean RerunsScience35.333.1−2.2
CharXiv (descriptive and reasoning combined)Multimodal93.295.4+2.2
CharXiv Extended (Mercor held-out)Multimodal76.778.9+2.2
SciCode 1.0.1 (Artificial Analysis)Code54.456.6+2.2
Terminal-Bench 4.0 (Artificial Analysis)Code21.719.7−2.0
WeirdML (v2)Science82.984.8+1.9
Tax Agent BenchFinance & Accounting64.966.8+1.9
Vision Arena OverallMultimodal12861293+1.8
SEA-HELMGeneral QA75.677.3+1.7
ArXivMath 04/2026Math62.564.2+1.7
HealthBenchHealth & Medicine58.857.1−1.7
Finance Agent v2 - General Qualitative AnalysisFinance & Accounting66.468.1+1.7
AIME 2025Math98.396.7−1.6
RedlineBenchLegal39.741.3+1.6
LiveCodeBench (Vals Public) - MediumCode89.891.4+1.6
GameBench 2Games62.564.0+1.5
GDPval Extended (Mercor held-out rubric score)Professional Work58.960.4+1.5
Terminal-Bench Science 0.1 (Vals)Science10.08.6−1.4
Vals Index v2Indexes60.962.3+1.3
VCT (SecureBio)Science48.246.9−1.3
HealthBench ProfessionalHealth & Medicine60.359.0−1.3
Arena Alignment IndexSafety74.575.8+1.3
MedScribeHealth & Medicine85.884.5−1.3
AA-AnalystAgent (pass^5)Professional Work45.046.3+1.3
SkillsBench with SkillsAgents59.258.0−1.2
MineBench — Bradley-Terry RatingGames17861806+1.2
TaxEval v2Finance & Accounting75.674.4−1.2
ForecastBench Preliminary Dataset Brier IndexForecasting61.660.5−1.1
Epoch Capabilities Index (ECI)Indexes160159−1.0
GDPval (Mercor 213-task rubric score)Professional Work84.583.5−1.0
Artificial Analysis Intelligence Index v4.3.2Indexes41.840.9−0.9
GDPval-AA v2.1Professional Work14611442−0.9
BrokenArXiv 04/2026Math34.433.6−0.8
Artificial Analysis Intelligence Index v4.3Indexes42.041.2−0.8
GDP.pdfProfessional Work24.023.2−0.8
DTBenchGeneral QA94.995.7+0.8
ACCoRDGeneral QA77.478.2+0.8
HPCT (SecureBio)Science63.162.4−0.7
MMLU ProGeneral QA90.090.7+0.7
VCT v2 (SecureBio)Science45.945.2−0.7
ObviousBenchGeneral QA99.3100.0+0.7
DistressBenchSafety71.470.7−0.7
Artificial Analysis Progress Index v1Indexes318320+0.7
SpatialBench V0 VerifiedScience66.467.0+0.6
OTIS Mock AIME 2024-2025Math98.398.9+0.6
PG-LLM-50Science0.40.4+0.5
Harvey's Legal Agent BenchmarkLegal9.610.0+0.4
AA-Omniscience IndexIndexes28.829.6+0.4
BioMysteryBench - Human SolvableScience88.588.8+0.3
Vals Index v2.1Indexes55.154.9−0.2
LiveCodeBench (Vals Public) - EasyCode99.199.10.0
ProgramBenchCode1.01.00.0

Opus 4.8vsGemini 3.8 Flashnormalised score marginsData Nov 15, 2026Data: Vals AI, Mercor, Artificial Analysis and 51 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…