Atlas

Compare Models

Claude Opus 5 vs Gemini 3.8 Flash

Claude Opus 5 leads by 3.4 capability points (≈95% interval +1.1 to +5.7): a clear lead over Gemini 3.8 Flash · P(higher) ≈ 100%

Wins 173 of 275 shared benchmarks (76 lost, 26 tied)

Capability vs. relative cost
Claude Opus 5Gemini 3.8 FlashClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Opus 5vsGemini 3.8 Flashcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins173 – 76 · 275 shared, 26 tied

Comparing matching setups only (54 benchmarks excluded) · suites counted once
← Claude Opus 5Gemini 3.8 Flash →

Claude Opus 5vsGemini 3.8 Flashwins by categoryData Nov 15, 2026Data: Vals AI, Mercor, Artificial Analysis and 71 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Gemini 3.8 Flash minus Claude Opus 5.

HPCT Refusal (CAIS)Safety100.013.0−87.0
BrokenArXiv 06/2026Math90.720.8−69.9
FrontierMath Tier 4 (v2) - PrivateMath73.222.0−51.2
ProofBench v1.1Math99.048.0−51.0
Olam Social Poker EloGames15181479−42.4
ChessBench EloGames8261527+41.7
Terminal-Bench 4.0Code53.913.1−40.8
LLM Creative Story WritingChat & Writing3.70.1−40.5
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.040.0+40.0
HieroglyphBenchMultimodal14.654.6+40.0
BlenderWorldCode73.034.2−38.8
Medical Long Context Reasoning (MLCR-AA)Health & Medicine59.421.7−37.8
Vending-Bench 2Games111825094−36.9
MASKSafety93.458.6−34.8
FrontierSWE V2Code52.019.6−32.4
ChessBench CoherenceGames63.894.6+30.8
Machiavelli Deception (CAIS)Safety96.586.5+30.1
LLM Debate BenchmarkChat & Writing17241514−29.9
Furniture Assembly BenchmarkMultimodal60.831.7−29.2
SWE-bench Verified Extended (Mercor held-out)Code82.054.0−28.0
IOI (Vals v2)Code84.356.9−27.4
MetagenomicsBench V1Science54.026.7−27.3
APEX-SWECode63.736.4−27.3
MetagenomicsBench V0Science54.026.7−27.3
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine40.013.3−26.7
Terminal-Bench 4.0 (Artificial Analysis)Code46.019.7−26.3
Hedge-BenchFinance & Accounting29.04.7−24.3
ScBench V1 Clean RerunsScience58.334.0−24.3
HLE Extended (Mercor held-out)General QA61.036.8−24.2
Document ArenaProfessional Work15091482−23.9
VCT-Refusal (CAIS)Safety29.46.4−23.0
BioSecBench-Refusal V1 Clean RerunsSafety24.547.4+22.9
HANDBOOK.mdProfessional Work32.310.8−21.5
Shroombench (mushroom identification, top-1)Multimodal44.065.0+21.0
Biopharma Bench V0.1Health & Medicine67.646.6−21.0
Code MigrationCode57.536.5−20.9
GameBench 2Games84.864.0−20.8
AA-Briefcase EloAgents15571202−20.6
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.020.0+20.0
ARC-AGI-3 (Semi-Private)General QA30.210.4−19.8
Creative Writing v3 EloChat & Writing21351744−19.8
VariantBench V1 Clean RerunsScience50.631.4−19.2
LMArena Agent Arena Net ImprovementAgents9.53.8−19.1
Concept Synth INDUCTION Challenge64 Round 1General QA32.814.1−18.7
Terminal-Bench Science 0.1 (Vals)Science27.18.6−18.6
AA-LCR Extended (Mercor held-out)Search73.955.6−18.3
Terminal-Bench-Science 0.1Science28.610.5−18.1
Bio Capabilities Index (SecureBio)Indexes178153−17.9
Artificial Analysis Coding Agent Index v1.5Indexes59.741.9−17.9
FrontierMath Tiers 1-3 (v2) - PrivateMath85.668.4−17.2
BioMysteryBench (Vals Terminus 2)Science79.362.2−17.0
SWE-bench VerifiedCode97.080.0−17.0
RiemannBenchMath68.051.2−16.8
Legal Research BenchLegal55.338.9−16.3
HPCT (SecureBio)Science78.662.4−16.2
BioTIER RefusalSafety100.083.9−16.1
CadQueryEval (Corrected Geometry)Code80.096.0+16.0
SciUniverse Level 1Science30.514.6−15.9
Humanity's Last Exam DiamondGeneral QA44.829.1−15.7
ALE-BenchCode26222182−15.6
HiL-Bench (Human-in-Loop Benchmark)Agents57.041.5−15.5
MedCodeHealth & Medicine63.648.1−15.4
Arena Image-to-WebDevCode16641594−15.3
Concept Synth INDUCTION Challenge100 Round 1General QA24.09.0−15.0
Chess Puzzles (task version 2.0)Games43.058.0+15.0
ORCA-bench Hard RCA (Fireworks 324-task subset)Agents44.329.5−14.8
LiveBench 2.0Agents82.167.3−14.7
Vals RSI Index v1.1 (historical five tasks)Agents32.117.6−14.5
Physical-Engineering's Last Exam (hidden test)Professional Work42.628.2−14.4
PLE (Physical-Engineering's Last Exam)Professional Work42.628.2−14.4
HealthBench ProfessionalHealth & Medicine73.459.0−14.4
SAGEMath49.435.1−14.4
LMArena Code Arena WebDevCode16911583−14.2
Eval Connections One-Shot with TrapsGames74.088.0+14.0
TaxCalcBench TY25Finance & Accounting20.06.0−14.0
Agent Red Teaming k=100 (CAIS)Safety43.329.8+13.5
Vals RSI Index v1.1 (four tasks)Agents33.019.9−13.1
Hemingway-benchChat & Writing10471093+12.7
MA-BenchProfessional Work56.243.6−12.6
FrontierCode 2.0 (Artificial Analysis)Code31.218.7−12.4
RSI-Exam 0.1 (Full 88)Agents0.50.3−12.3
Conceptual Reasoning IndexIndexes74.765.7−12.3
MineBench — Bradley-Terry RatingGames20081806−12.2
Vals Science IndexIndexes47.935.8−12.2
Mystery Game PuzzlesGames59.047.0−12.0
LVBenchMultimodal75.487.1+11.7
WCB (SecureBio)Science68.857.1−11.7
Public Benefits Bench v1.1Professional Work76.965.3−11.6
WeirdML v3Science19.07.4−11.6
LMCAGeneral QA64.552.9−11.5
CVDP (517-problem set)Code62.250.7−11.5
Antidote: Everyday EditionSafety10551087+11.4
DeepsecBench-AA (Artificial Analysis)Cybersecurity25.914.5−11.3
MBCT (SecureBio)Science60.248.9−11.3
ComplexConstraintsChat & Writing37.348.4+11.1
Humanity's Sixth SenseMultimodal30.541.6+11.1
SnorkelFinance 2.0Finance & Accounting16.55.4−11.1
GeneBench-Pro Public Release SubsetScience22.011.0−11.0
CritPt (avg@5)Science29.118.3−10.9
ChessBench AccuracyGames71.081.1+10.8
LLM Chess EloGames12851546+10.7
EnterpriseBench: CoreCraftProfessional Work68.758.5−10.2
Humanity's Last ExamGeneral QA56.446.2−10.2
MaxxBenchCode63.553.4−10.1
FrontierCode 1.1 Extended - Pass RateCode69.659.6−10.0
HealthBenchHealth & Medicine67.157.1−10.0
Tax Agent BenchFinance & Accounting76.766.8−9.9
Vibe Code Bench v1.1Code88.478.7−9.8
PCT Political Inconsistency (CAIS)Safety36.746.3−9.6
AutoResearchExam (hidden-test AUARC)Agents0.60.5−9.6
FrontierPhysics v0.1Science9.50.0−9.5
Longform Creative WritingChat & Writing86.376.8−9.5
GBENCH One-shot Coding GScoreGames68.158.6−9.5
LMArena Agent Arena Net Improvement (Steering Burden)Agents7.22.3−9.3
SpatialBench V1 Clean RerunsScience68.459.4−9.0
Roboflow Vision Evals (October 2026, five tasks)Multimodal70.979.8+8.9
ERQAMultimodal70.879.6+8.8
Bug Hunt BenchCode25.717.1−8.6
DAYJOB: FinanceFinance & Accounting11.32.8−8.5
TxBench Preclinical Pharmacology V1 Clean RerunsScience63.054.7−8.3
Blueprint-Bench 2Multimodal0.30.4+8.2
GDPval-AA v2Professional Work16291464−8.1
Bedside BenchHealth & Medicine94.686.5−8.1
MindCube TinyMultimodal84.392.2+7.9
Snorkel SWE-bench CLICode14.06.3−7.7
ATLAS-Finance v1.0Finance & Accounting12.34.7−7.7
DAYJOB: HealthcareHealth & Medicine8.40.8−7.6
AA-AnalystAgent (pass^5)Professional Work53.846.3−7.5
BioSecBench-Surveillance V0 (102 tasks)Science51.544.1−7.4
ScBench-Long V1 Clean RerunsScience27.319.9−7.4
BioSecBench-Surveillance V1 Clean RerunsScience51.544.1−7.4
FrontierCode 1.1 Extended - ScoreCode63.656.3−7.3
BioSecBench-RespiratoryVirus-Surveillance V1 (110 evaluations)Science14.67.3−7.3
GDPval-AA v2.1Professional Work16051442−7.2
LMArena Agent Arena Bash RecoveryAgents13.6-0.7−7.1
ATLAS-Finance v1.1Finance & Accounting10.73.7−7.0
WeirdML (v2)Science91.884.8−7.0
TxBench Oligo Discovery V1 Clean RerunsScience40.033.1−6.9
SnorkelRevOpsProfessional Work12.25.4−6.8
Roboflow Vision Evals (September 2026 protocol)Multimodal78.385.1+6.8
MortgageTax (Vals all 1,258 documents)Finance & Accounting72.165.3−6.7
EnigmaEvalGeneral QA43.937.2−6.7
BioSecBench-Function V1 Clean RerunsScience42.536.0−6.5
MedScribeHealth & Medicine91.084.5−6.5
Terminal-Bench 2.1 Extended (Mercor held-out)Code38.044.4+6.4
Snorkel Agentic Coding 2.0Code38.932.6−6.3
CritPt v2 (agentic, avg@5)Science71.465.1−6.3
TextQuests Average Cumulative HarmSafety22.624.4−6.0
NoCoT-Bench NCRI 15.2General QA115126+5.9
ACCoRDGeneral QA83.878.2−5.7
SM BenchSafety85.880.4−5.4
MacroscopeBenchCode65.570.7+5.2
CrossSearch (pass^3)Search34.029.0−5.0
VIALSMultimodal24.429.4+5.0
MedXpertQA MM (200-question set)Health & Medicine79.884.7+4.9
CUA-benchAgents9.04.2−4.8
Snorkel Digital Work IndexIndexes21.816.9−4.8
Artificial Analysis Progress Index v1Indexes334320−4.7
Epoch Capabilities Index (ECI)Indexes164159−4.7
VCT (SecureBio)Science51.446.9−4.5
LMArena Agent Arena Confirmed SuccessAgents10.21.3−4.4
Artificial Analysis Intelligence IndexIndexes63.158.7−4.4
Harvey LAB: In-House Contracting (held-out)Legal2.06.3+4.3
AutomationBench-AA 1.0.6Agents56.660.9+4.3
WorldAuditBenchMultimodal28.232.4+4.2
Realm Financial ReasoningFinance & Accounting47.451.5+4.1
micro1 PharmaBenchHealth & Medicine9.75.6−4.1
APEX-Agents 1.1Professional Work77.072.9−4.1
EpiBench V1 Clean RerunsScience25.221.4−3.8
AA-Omniscience IndexIndexes37.129.6−3.8
Snorkel WorkplaceAgentsProfessional Work14.911.2−3.7
Design Arena Overall Code CategoriesCode13211299−3.6
LMArena Agent Arena Praise vs ComplaintAgents12.24.9−3.6
Epiq LAER CorporateBenchProfessional Work74.077.6+3.6
RuneBenchGames5.76.0+3.6
Creative Writing v3 Rubric ScoreChat & Writing16.315.6−3.3
ProgramBench (Mercor 100-task subset)Code4.31.0−3.3
AIME 2025Math100.096.7−3.3
BuseyBench SVGMultimodal6.36.6+3.3
MedXpertQA MM Extended (Mercor held-out)Health & Medicine64.160.8−3.3
LMArena Agent Arena SteerabilityAgents10.74.3−3.2
Extended NYT Connections 940 (Quadratic v1)Games94.397.4+3.1
SciCode (main problems)Science17.314.2−3.1
Toloka Arena (v3)Agents62.059.0−3.0
LAB-Bench ProtocolQA (SecureBio)Science81.184.0+2.9
LMArena Agent Arena Steering BurdenAgents4.4-1.3−2.8
APEX-AccountingFinance & Accounting54.551.7−2.8
AutomationBench 1.0.6 (Zapier)Agents26.929.7+2.7
Political Science LLM BenchmarkProfessional Work71.468.7−2.7
RedlineBenchLegal44.041.3−2.7
IntPhys 2Multimodal65.162.5−2.6
GPQA DiamondGeneral QA92.494.9+2.5
BrowseComp Extended (Mercor held-out)Search44.041.5−2.5
CharXiv ReasoningMultimodal83.786.2+2.5
GDPval Extended (Mercor held-out rubric score)Professional Work62.960.4−2.5
Harvey HLAB — Criteria Pass RateLegal87.790.2+2.5
SpatialViz-BenchMultimodal75.878.1+2.3
SnorkelManufacturingProfessional Work9.47.1−2.3
DTBenchGeneral QA97.995.7−2.2
SUPERChemScience74.772.6−2.1
MMMU-Pro Extended (Mercor held-out)Multimodal34.336.4+2.1
GDPval (Mercor 213-task rubric score)Professional Work85.683.5−2.1
PG-LLM-50Science0.40.4−2.0
AA-LCR v1.1Search82.084.0+2.0
GDP.pdf - all-pass rate reported by GoogleProfessional Work37.035.0−2.0
WordleBenchGames96.098.0+2.0
Vision Arena OverallMultimodal12851293+2.0
SpeciEval (Pooled English Runs)Safety87.989.9+1.9
SpeciEval (Refusal-Aware Retries, Historical)Safety87.989.9+1.9
SimpleBenchGeneral QA80.682.4+1.8
BrowseComp (Mercor 130-question subset)Search84.683.1−1.5
ForecastBench Preliminary Dataset Brier IndexForecasting62.060.5−1.5
Clipping Bench v1.0Games734596−1.5
TxBench Antibody Discovery V1 Clean RerunsScience41.943.3+1.5
Excel Modeling BenchmarkFinance & Accounting73.672.2−1.4
LMArena Text Arena EloChat & Writing14901497+1.3
MMLU ProGeneral QA92.090.7−1.3
ARC-AGI-2General QA90.489.2−1.3
SnorkelLegalLegal26.425.3−1.1
DAXBench v2Code86.287.2+1.1
𝜏³-BankingAgents44.745.8+1.0
ARC-AGI-1 Semi-PrivateGeneral QA97.598.5+1.0
DeepSWE v1.1Code72.873.8+1.0
MysteryMechanismScience37.436.5−0.9
CRM-Agent-BenchAgents82.783.6+0.9
CharXiv Extended (Mercor held-out)Multimodal79.878.9−0.9
NoCoT-Bench NCKI (kspine v3)General QA119121+0.9
CharXiv (descriptive and reasoning combined)Multimodal94.695.4+0.8
GDP.pdfProfessional Work24.023.2−0.8
SnorkelUnderwrite 2.0Finance & Accounting18.319.1+0.8
TextQuests Game ProgressGames53.354.1+0.8
TaxEval v2Finance & Accounting75.174.4−0.7
SpatialBench-Long V1 Clean RerunsScience28.028.6+0.6
DistressBenchSafety70.270.7+0.5
MMMU ProMultimodal84.084.5+0.5
LiveCodeBench (Vals AI public dataset)Code89.089.5+0.5
SWE-TogetherCode63.864.2+0.4
SpatialBench V0 VerifiedScience66.767.0+0.3
LMArena Agent Arena Tool HallucinationAgents0.90.4−0.2
SciCode 1.0.1 (Artificial Analysis)Code56.456.6+0.2
Arena Alignment IndexSafety75.675.8+0.2
APEX-1 (Mercor, mean score)Professional Work68.668.8+0.2
LAB-Bench LitQA2 (SecureBio)Science51.451.5+0.1
LMArena Agent Arena Bash ReliabilityAgents9.89.9+0.1
LegalBenchLegal87.087.00.0
BabaIsBench Stage 0: The IntroGames100.0100.00.0
FrontierMath ErdősMath0.00.00.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal20.020.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.00.00.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal20.020.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal20.020.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
LAB-Bench CloningScenarios (SecureBio)Science100.0100.00.0
Next.js AI Agent EvaluationsAgents100.0100.00.0
ObviousBenchGeneral QA100.0100.00.0
OTIS Mock AIME 2024-2025Math98.998.90.0
Terminal-Bench 2.1Code87.687.60.0
Time Horizon Index: KSPAgents18.818.80.0

Claude Opus 5vsGemini 3.8 Flashnormalised score marginsData Nov 15, 2026Data: Vals AI, Mercor, Artificial Analysis and 71 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…