Atlas

Compare Models

Claude Opus 5 vs Gemini 3.7 Flash

Claude Opus 5 leads by 4.7 capability points (≈95% interval +2.5 to +6.9): a clear lead over Gemini 3.7 Flash · P(higher) ≈ 100%

Wins 174 of 255 shared benchmarks (62 lost, 19 tied)

Capability vs. relative cost
Claude Opus 5Gemini 3.7 FlashClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Opus 5vsGemini 3.7 Flashcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins174 – 62 · 255 shared, 19 tied

Comparing matching setups only (15 benchmarks excluded) · suites counted once
← Claude Opus 5Gemini 3.7 Flash →

Claude Opus 5vsGemini 3.7 Flashwins by categoryData Nov 15, 2026Data: Vals AI, Arena, Mercor and 62 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Gemini 3.7 Flash minus Claude Opus 5.

HPCT Refusal (CAIS)Safety100.013.0−87.0
BrokenArXiv 06/2026Math90.711.1−79.6
ATLAS-Education (StudentBench) Lesson PlanningProfessional Work1.1-0.4−62.8
Olam Social Poker EloGames15181468−54.3
Odysseys2Agents58.35.8−52.5
SWE-bench Verified Extended (Mercor held-out)Code82.030.0−52.0
LLM Creative Story WritingChat & Writing3.7-0.7−50.2
Terminal-Bench 4.0Code53.96.1−47.9
Medical Long Context Reasoning (MLCR-AA)Health & Medicine59.415.0−44.4
ProofBench v1.1Math99.058.0−41.0
UnityBench-Mini3DCode71.831.2−40.6
HieroglyphBenchMultimodal14.655.0+40.4
MASKSafety93.453.7−39.7
LLM Debate BenchmarkChat & Writing17241455−38.3
ChessBench — Historical Output RepairGames32.770.7+38.0
FrontierMath Tier 4 (v2) - PrivateMath73.236.6−36.6
LMArena Agent Arena Net ImprovementAgents9.5-1.3−35.8
Furniture Assembly BenchmarkMultimodal60.826.7−34.2
Machiavelli Deception (CAIS)Safety96.585.5+33.2
Terminal-Bench 4.0 (Artificial Analysis)Code46.013.6−32.3
FrontierSWE V2Code52.020.3−31.8
BioSecBench-Refusal V1 Clean RerunsSafety24.554.8+30.3
HLE Extended (Mercor held-out)General QA61.031.6−29.4
RiemannBenchMath68.039.2−28.8
Document ArenaProfessional Work15091476−28.8
SWE-Marathon v1.1Code50.022.5−27.5
ALE-BenchCode26221876−26.4
ChessBench EloGames8261265+26.1
AA-Briefcase EloAgents15571116−25.6
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine40.014.7−25.3
AA-LCR Extended (Mercor held-out)Search73.948.9−25.0
VCT-Refusal (CAIS)Safety29.44.8−24.6
Continual Learning Exam (Trial 1)Agents42.218.7−23.5
Learn2Play BenchGames81.858.3−23.5
Chess Puzzles (task version 2.0)Games43.066.0+23.0
Code MigrationCode57.534.8−22.7
MetagenomicsBench V1Science54.031.7−22.3
MetagenomicsBench V0Science54.031.7−22.3
Mystery Game PuzzlesGames59.037.0−22.0
APEX-SWECode63.742.0−21.7
HANDBOOK.mdProfessional Work32.310.8−21.5
Creative Writing v3 EloChat & Writing21351720−21.0
Legal Research BenchLegal55.334.6−20.7
Senior SWE-BenchCode34.714.7−20.0
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal20.040.0+20.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal20.00.0−20.0
Terminal-Bench Science 0.1 (Vals)Science27.17.1−20.0
ChessBench CoherenceGames63.883.2+19.4
VariantBench V1 Clean RerunsScience50.631.4−19.2
Tax Agent BenchFinance & Accounting76.757.7−19.0
Arena Code WebDev — Brand & MarketingCode16811553−18.6
BrowseComp Extended (Mercor held-out)Search44.025.5−18.5
Vibe Code Bench v1.1Code88.470.4−18.0
LMArena Agent Arena Net Improvement (Steering Burden)Agents7.2-2.2−17.8
iPrefBench (task success)Agents47.730.0−17.7
Arena Code WebDev — ReactCode17131584−17.3
Shroombench (mushroom identification, top-1)Multimodal44.061.0+17.0
IOI (Vals v2)Code84.367.8−16.5
SWE-bench VerifiedCode97.080.8−16.2
Hemingway-benchChat & Writing10471105+16.1
Concept Synth INDUCTION Challenge64 Round 1General QA32.817.2−15.6
BioTIER RefusalSafety100.084.4−15.6
GBENCH One-shot Coding GScoreGames68.152.6−15.5
RSI-Exam 0.1 (Full 88)Agents0.50.3−15.5
Arena Code WebDev — FrontendCode17121592−15.4
Arena Code WebDev — SimulationsCode17511618−15.1
LiveBench 2.0Agents82.167.2−14.9
CritPt (avg@5)Science29.114.3−14.9
Vals RSI Index v1.1 (four tasks)Agents33.018.3−14.8
MineBench — Bradley-Terry RatingGames20081765−14.7
HackerRank-BenchCode42.628.5−14.1
LMCAGeneral QA64.550.4−14.1
Arena Code WebDev — GamingCode17641638−14.1
FrontierMath Tiers 1-3 (v2) - PrivateMath85.671.6−14.0
Arena Code WebDev — Content Creation ToolsCode16621574−13.9
HPCT (SecureBio)Science78.664.7−13.9
Conceptual Reasoning IndexIndexes74.764.6−13.9
Arena Image-to-WebDevCode16641600−13.9
Humanity's Last Exam DiamondGeneral QA44.831.2−13.6
Arena Code WebDev — Reference-Based DesignCode17081608−13.3
Concept Synth INDUCTION Challenge100 Round 1General QA24.011.0−13.0
EnterpriseBench: CoreCraftProfessional Work68.755.9−12.8
DeepResearch Bench II (Mercor 130-task set)Search56.143.4−12.7
ScBench-Long V0 VerifiedScience36.424.2−12.1
Wiz Cyber Model Arena v3.9Cybersecurity71.259.1−12.1
Arena Code WebDev — HTMLCode16791593−12.0
Artificial Analysis Intelligence Index v4.3.2Indexes50.839.1−11.7
Bio Capabilities Index (SecureBio)Indexes178162−11.6
Artificial Analysis Intelligence Index v4.3Indexes50.739.4−11.3
WCB (SecureBio)Science68.858.0−10.8
Humanity's Last ExamGeneral QA56.445.6−10.8
MedCodeHealth & Medicine63.653.4−10.2
Arena Code WebDev — FullstackCode16681629−10.0
BioSecBench-Surveillance V1 Clean RerunsScience51.541.5−10.0
BioSecBench-Surveillance V0 (102 tasks)Science51.541.5−10.0
EnigmaEvalGeneral QA43.933.9−10.0
LVBenchMultimodal75.485.4+10.0
GDPval-AA v2Professional Work16291433−9.6
LMArena Agent Arena Bash RecoveryAgents13.6-5.4−9.5
AutoformBenchMath61.952.5−9.4
Humanity's Sixth SenseMultimodal30.539.8+9.3
SpatialBench V1 Clean RerunsScience68.459.1−9.3
𝜏³-BankingAgents44.735.5−9.3
Roboflow Vision Evals (October 2026, five tasks)Multimodal70.980.1+9.2
SpatialBench-Long V0 VerifiedScience31.822.7−9.1
GDPval-AA v2.1Professional Work16051404−8.8
BioSecBench-Function V1 Clean RerunsScience42.533.7−8.8
ChessBench AccuracyGames71.079.2+8.8
ScBench-Long V1 Clean RerunsScience27.318.8−8.5
DAYJOB: FinanceFinance & Accounting11.32.8−8.5
Harvey LAB (Mercor run, 1,749 public tasks)Legal17.89.3−8.5
PCT Political Inconsistency (CAIS)Safety36.745.1−8.4
DAYJOB: HealthcareHealth & Medicine8.40.0−8.4
GeneBench-Pro Public Release SubsetScience22.014.0−8.0
ScBench V1 Clean RerunsScience58.350.4−7.9
TxBench Preclinical Pharmacology V1 Clean RerunsScience63.055.3−7.7
LMArena Agent Arena Praise vs ComplaintAgents12.2-2.9−7.5
SimpleVQA with toolsMultimodal70.377.8+7.5
ATLAS-Finance v1.1Finance & Accounting10.73.3−7.3
LLM Chess EloGames12851107−7.3
FrontierCode 1.1 Extended - Pass RateCode69.662.3−7.3
Arena Code WebDev — Consumer ProductCode16321574−7.3
FrontierChallenge (Pass Rate)Science17.510.3−7.2
Antidote: Everyday EditionSafety10551075+7.1
APEX-Accounting 1.1Finance & Accounting64.357.2−7.1
MedScribeHealth & Medicine91.083.9−7.0
Roboflow Vision Evals (September 2026 protocol)Multimodal78.385.2+6.9
ERQAMultimodal70.877.7+6.9
ACCoRDGeneral QA83.877.0−6.9
LMArena Agent Arena Confirmed SuccessAgents10.2-3.5−6.8
CyberArenaCybersecurity73.867.1−6.7
Terminal-Bench 2.1 Extended (Mercor held-out)Code38.044.4+6.4
AA-AnalystAgent (pass^5)Professional Work53.860.0+6.3
NoCoT-Bench NCRI 15.2General QA115126+6.0
Eval Connections One-Shot with TrapsGames74.080.0+6.0
TaxCalcBench TY25Finance & Accounting20.014.0−6.0
TextQuests Average Cumulative HarmSafety22.624.4−6.0
LMArena Agent Arena Steering BurdenAgents4.4-7.3−5.9
ARC-AGI-2General QA90.484.6−5.8
GlobeBenchGeneral QA15.921.8+5.8
Artificial Analysis Progress Index v1Indexes334317−5.7
LAB-Bench ProtocolQA (SecureBio)Science81.186.8+5.7
AutomationBench-AA 1.0.6Agents56.662.0+5.5
MortgageTax (Vals all 1,258 documents)Finance & Accounting72.166.7−5.4
BrowseComp (Mercor 130-question subset)Search84.679.2−5.4
LMArena Agent Arena SteerabilityAgents10.7-0.0−5.3
AA-Omniscience IndexIndexes37.126.5−5.3
Creative Writing v3 Rubric ScoreChat & Writing16.315.2−5.2
MBCT (SecureBio)Science60.255.0−5.2
SM BenchSafety85.880.6−5.2
ComplexConstraintsChat & Writing37.342.4+5.1
GDPval Extended (Mercor held-out rubric score)Professional Work62.957.8−5.1
Epoch Capabilities Index (ECI)Indexes164158−4.9
Agent Red Teaming k=100 (CAIS)Safety43.338.5+4.8
Bug Hunt BenchCode25.721.0−4.8
Judgemark v4Chat & Writing78.883.4+4.6
TxBench Oligo Discovery V1 Clean RerunsScience40.035.6−4.4
MindCube TinyMultimodal84.388.4+4.1
Arena Code WebDev — Data & AnalyticsCode16041577−4.0
Toloka Arena (v3)Agents62.058.0−4.0
CadQueryEval (Corrected Geometry)Code80.084.0+4.0
MedXpertQA MM (200-question set)Health & Medicine79.883.5+3.7
RuneBenchGames5.76.0+3.6
MRCR v2 (8-needle, Context Arena) - 128K AUCGeneral QA97.593.9−3.6
PG-LLM-50Science0.40.3−3.5
SciCode 1.0.1 (Artificial Analysis)Code56.459.8+3.5
DeepSWE v1.1Code68.965.5−3.4
BuseyBench SVGMultimodal6.36.6+3.4
ProgramBench (Mercor 100-task subset)Code4.31.0−3.3
AIME 2025Math100.096.7−3.3
EpiBench V1 Clean RerunsScience25.222.0−3.1
IntPhys 2Multimodal65.168.2+3.1
GPQA DiamondGeneral QA92.495.5+3.0
GDP.pdf - all-pass rate reported by GoogleProfessional Work37.034.0−3.0
EnterpriseOps-Gym-AAAgents47.550.4+2.9
BioSecBench-Function V0Science43.040.1−2.9
Design Arena Overall Code CategoriesCode13211304−2.8
LAB-Bench LitQA2 (SecureBio)Science51.454.2+2.8
MRCR v2 (8-needle, Context Arena) - 128K pointwiseGeneral QA91.388.6−2.7
TxBench Oligo Discovery V0Science42.239.5−2.6
APEX-Agents 1.1Professional Work77.079.4+2.4
SpatialBench V0 VerifiedScience66.764.3−2.3
Agents' Last Exam - release-reported pass rate, subset unspecifiedAgents28.626.3−2.3
Excel Modeling BenchmarkFinance & Accounting73.671.3−2.2
Vision Arena OverallMultimodal12851294+2.1
LABBench2Science84.282.1−2.1
MMMU-Pro Extended (Mercor held-out)Multimodal34.336.4+2.1
VCT (SecureBio)Science51.449.3−2.1
VIALSMultimodal24.426.5+2.1
ARC-AGI-1 Semi-PrivateGeneral QA97.595.5−2.0
SUPERChemScience74.772.8−1.9
Terminal-Bench 2.1Code87.685.8−1.9
LMArena Agent Arena Bash ReliabilityAgents9.86.2−1.8
SpatialBench-Long V1 Clean RerunsScience28.029.8+1.8
Toloka Arena (v1)Agents52.954.6+1.7
TxBench Antibody Discovery V1 Clean RerunsScience41.943.5+1.7
OTIS Mock AIME 2024-2025Math98.997.2−1.7
APEX-1 (Mercor, mean score)Professional Work68.667.0−1.6
MMLU ProGeneral QA92.090.6−1.4
MedXpertQA MM Extended (Mercor held-out)Health & Medicine64.162.7−1.4
TxBench Antibody Discovery V0Science51.450.0−1.4
RedlineBenchLegal44.045.3+1.3
DistressBenchSafety70.269.0−1.3
NoCoT-Bench NCKI (kspine v3)General QA119121+1.2
Arena Alignment IndexSafety75.674.4−1.2
SpeciEval (Pooled English Runs)Safety87.989.0+1.1
SpeciEval (Refusal-Aware Retries, Historical)Safety87.989.0+1.1
GPQA-Bio (SecureBio)Science82.781.6−1.1
DTBenchGeneral QA97.996.8−1.1
AA-LCR v1.1Search82.083.0+1.0
WordleBenchGames96.097.0+1.0
Harvey HLAB — Criteria Pass RateLegal87.788.6+0.9
MMMU ProMultimodal84.084.9+0.9
CharXiv ReasoningMultimodal83.784.5+0.8
ATLAS-Education (StudentBench) Learning GainProfessional Work12.614.1+0.7
CharXiv (descriptive and reasoning combined)Multimodal94.695.3+0.7
CharXiv Extended (Mercor held-out)Multimodal79.879.1−0.7
LAB-Bench CloningScenarios (SecureBio)Science100.099.4−0.6
SpatialViz-BenchMultimodal75.875.3−0.5
TextQuests Game ProgressGames53.352.9−0.4
TaxEval v2Finance & Accounting75.174.7−0.4
ForecastBench Tournament Overall Brier IndexForecasting66.666.2−0.4
GDPval (Mercor 213-task rubric score)Professional Work85.685.2−0.4
LiveCodeBench (Vals AI public dataset)Code89.088.7−0.4
MMLU-Bio (SecureBio)Science91.191.4+0.3
ForecastBench Preliminary Dataset Brier IndexForecasting62.061.7−0.3
LegalBenchLegal87.087.3+0.3
GDP.pdfProfessional Work24.023.8−0.2
SAGEMath49.449.2−0.2
LMArena Agent Arena Tool HallucinationAgents0.90.7−0.1
LMArena Text Arena EloChat & Writing14901489−0.1
BabaIsBench Stage 0: The IntroGames100.0100.00.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.00.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.00.00.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal20.020.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
ObviousBenchGeneral QA100.0100.00.0

Claude Opus 5vsGemini 3.7 Flashnormalised score marginsData Nov 15, 2026Data: Vals AI, Arena, Mercor and 62 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…