Atlas

Compare Models

Claude Opus 5.5 vs Gemini 3.7 Flash

Claude Opus 5.5 leads by 8.4 capability points (≈95% interval +6.0 to +10.7): a clear lead over Gemini 3.7 Flash · P(higher) ≈ 100%

Wins 164 of 213 shared benchmarks (28 lost, 21 tied)

Capability vs. relative cost
Claude Opus 5.5Gemini 3.7 FlashClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Opus 5.5vsGemini 3.7 Flashcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins164 – 28 · 213 shared, 21 tied

Comparing matching setups only (11 benchmarks excluded) · suites counted once
← Claude Opus 5.5Gemini 3.7 Flash →

Claude Opus 5.5vsGemini 3.7 Flashwins by categoryData Nov 15, 2026Data: Vals AI, Mercor, Arena and 52 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Gemini 3.7 Flash minus Claude Opus 5.5.

BrokenArXiv 06/2026Math100.011.1−88.9
HPCT Refusal (CAIS)Safety100.013.0−87.0
BrokenArXiv 05/2026Math95.08.5−86.5
VCT-Refusal (CAIS)Safety90.44.8−85.6
Terminal-Bench 4.0Code64.86.1−58.8
FrontierMath Tier 4 (v2) - PrivateMath95.036.6−58.4
Furniture Assembly BenchmarkMultimodal83.326.7−56.7
Concept Synth INDUCTION Challenge64 Round 1General QA71.917.2−54.7
Concept Synth INDUCTION Challenge100 Round 1General QA65.011.0−54.0
Medical Long Context Reasoning (MLCR-AA)Health & Medicine66.715.0−51.7
BioSecBench-Refusal V1 Clean RerunsSafety3.854.8+51.0
LLM Creative Story WritingChat & Writing3.7-0.7−50.0
Terminal-Bench 4.0 (Artificial Analysis)Code56.613.6−42.9
Terminal-Bench Science 0.1 (Vals)Science48.65.7−42.9
ProofBench v1.1Math100.058.0−42.0
FrontierSWE V2 Worst@5Code52.010.3−41.7
Olam Social Poker EloGames15061468−41.3
LMArena Agent Arena Net Improvement (Steering Burden)Agents19.5-2.2−41.0
FrontierSWE V2 Best@5Code71.531.2−40.3
BioSecBench-Function V1 Clean RerunsScience73.633.7−39.9
Humanity's Last Exam RMS Calibration Error (CAIS)General QA13.851.9−38.1
Arena Code WebDev — ReactCode18671584−37.9
MirrorCode (ML, +Private, 2L)Code78.941.1−37.8
TxBench Antibody Discovery V1 Clean RerunsScience6.843.5+36.7
Arena Code WebDev — Brand & MarketingCode18041553−36.6
SWE-bench Verified Extended (Mercor held-out)Code66.030.0−36.0
MASKSafety89.453.7−35.7
BrowseComp Extended (Mercor held-out)Search60.525.5−35.0
LLM Debate BenchmarkChat & Writing17011455−35.0
Arena Code WebDev — SimulationsCode19221618−34.4
ArXivMath 05/2026Math81.747.5−34.2
TxBench Oligo Discovery V1 Clean RerunsScience1.435.6+34.2
Mystery Game PuzzlesGames71.037.0−34.0
Arena Code WebDev — FrontendCode18491592−33.0
Arena Code WebDev — GamingCode19291638−32.5
AA-Briefcase v1.1 EloAgents16821115−31.1
ALE-BenchCode27481876−30.9
Arena Code WebDev — Content Creation ToolsCode17681574−30.8
RiemannBenchMath69.639.2−30.4
Arena Code WebDev — Reference-Based DesignCode18371608−30.3
TxBench Preclinical Pharmacology V1 Clean RerunsScience26.355.3+29.0
Arena Code WebDev — FullstackCode17391629−28.9
TaxCalcBench TY25Finance & Accounting42.014.0−28.0
APEX-SWECode67.642.0−25.6
AA-LCR Extended (Mercor held-out)Search74.248.9−25.3
DAYJOB: HealthcareHealth & Medicine24.70.0−24.7
Senior SWE-BenchCode39.314.7−24.6
Machiavelli Deception (CAIS)Safety77.585.5−24.3
EnigmaEvalGeneral QA57.833.9−23.9
Humanity's Last Exam DiamondGeneral QA54.631.2−23.4
HackerRank-BenchCode51.228.5−22.8
RSI-Exam 0.1 (Full 88)Agents0.50.3−22.7
MineBench — Bradley-Terry RatingGames21341765−22.4
Learn2Play BenchGames80.158.3−21.8
LMArena Agent Arena Praise vs ComplaintAgents40.3-2.9−21.6
GBENCH One-shot Coding GScoreGames73.952.6−21.3
DAYJOB: FinanceFinance & Accounting23.92.8−21.1
ArXivMath 06/2026Math83.362.5−20.8
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal20.040.0+20.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.020.0+20.0
FrontierMath Tiers 1-3 (v2) - PrivateMath91.271.6−19.6
Arena Code WebDev — HTMLCode17321593−19.5
LiveBench 2.0Agents86.567.2−19.3
LLM Chess EloGames15751107−19.2
VariantBench V1 Clean RerunsScience50.631.4−19.2
Vals RSI Index v1.1 (four tasks)Agents37.318.3−19.0
ProgramBench (Mercor 100-task subset)Code20.01.0−19.0
Bug Hunt BenchCode39.721.0−18.7
Humanity's Last Exam (Text-Only)General QA57.539.0−18.5
EnterpriseBench: CoreCraftProfessional Work74.455.9−18.5
Conceptual Reasoning IndexIndexes77.964.6−18.3
GDPval Extended (Mercor held-out rubric score)Professional Work75.857.8−18.0
LMCAGeneral QA68.250.4−17.8
HieroglyphBenchMultimodal37.455.0+17.6
CritPt (avg@5)Science31.714.3−17.4
SWE-bench VerifiedCode98.281.0−17.2
Toloka Arena (v3)Agents75.058.0−17.0
EpiBench V1 Clean RerunsScience39.022.0−17.0
LAB-Bench LitQA2 (SecureBio)Science71.154.2−16.9
Creative Writing v3 EloChat & Writing20511720−16.8
Arena Code WebDev — Consumer ProductCode17051574−16.5
Remote Labor Index (RLI)Professional Work21.35.0−16.2
CadQueryEval (Corrected Geometry)Code100.084.0−16.0
PCT Political Inconsistency (CAIS)Safety29.145.1−16.0
GeneBench-Pro Public Release SubsetScience30.014.0−16.0
Arena Code WebDev — Data & AnalyticsCode16801577−15.4
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine29.314.7−14.7
MMMU-Pro Extended (Mercor held-out)Multimodal51.036.4−14.6
SpatialViz-BenchMultimodal89.775.3−14.4
BioSecBench-Surveillance V1 Clean RerunsScience55.841.5−14.3
BioTIER RefusalSafety98.784.4−14.3
Agent Red Teaming k=100 (CAIS)Safety24.438.5−14.0
VIALSMultimodal40.226.5−13.7
MBCT (SecureBio)Science68.755.0−13.7
SpatialBench V1 Clean RerunsScience72.859.1−13.6
GDPval-AA v2.1Professional Work17091404−13.4
Design Arena Overall Code CategoriesCode13841304−13.3
Veracode GenAI Code Security (Fall 2026)Cybersecurity73.060.0−13.0
LMArena Agent Arena Steering BurdenAgents18.6-7.3−13.0
SWE-bench MultilingualCode95.582.6−12.9
Harvey HLAB — Criteria Pass RateLegal75.888.6+12.8
VCT (SecureBio)Science61.449.3−12.1
LMArena Agent Arena Confirmed SuccessAgents20.7-3.5−12.1
ScBench-Long V1 Clean RerunsScience6.818.8+11.9
VCT v2 (SecureBio)Science60.548.6−11.9
ACCoRDGeneral QA88.477.0−11.4
RuneBenchGames6.66.0−10.3
LMArena Agent Arena Bash RecoveryAgents15.0-5.4−10.2
AA-Omniscience IndexIndexes46.426.5−10.0
DeepResearch Bench II (Mercor 130-task set)Search53.343.4−9.9
SpatialBench-Long V1 Clean RerunsScience20.229.8+9.5
GPQA-Bio (SecureBio)Science90.981.6−9.3
BrowseComp (Mercor 130-question subset)Search88.579.2−9.3
FrontierCode 1.1 Extended - ScoreCode65.356.3−9.0
LABBench2Science73.182.1+9.0
ARC-AGI-2General QA93.384.6−8.7
Arena Alignment IndexSafety83.274.4−8.7
DeepSWE v1.1Code74.265.5−8.7
FrontierCS Algorithmic v1.1Code56.548.0−8.6
MedXpertQA MM Extended (Mercor held-out)Health & Medicine71.262.7−8.5
FrontierCode 1.1 Extended - Pass RateCode70.862.3−8.5
APEX-Accounting 1.1Finance & Accounting65.557.2−8.3
Epoch Capabilities Index (ECI)Indexes169158−8.3
Ramp Accounting BenchFinance & Accounting50.542.3−8.2
Gray Swan IPI Attack Success (k=15)Safety1.09.2−8.2
MetagenomicsBench V1Science39.331.7−7.7
MetagenomicsBench V0Science39.331.7−7.6
ComplexConstraintsChat & Writing49.942.4−7.5
MedScribeHealth & Medicine91.483.9−7.5
HardGeoBench Avg DistanceMultimodal655895−7.4
LAB-Bench ProtocolQA (SecureBio)Science94.086.8−7.2
SciCode 1.0.1 (Artificial Analysis)Code66.959.8−7.1
SUPERChemScience79.672.8−6.8
DistressBenchSafety75.669.0−6.6
HardGeoBench <=100miMultimodal62.556.3−6.2
xbench-ScienceQAScience76.470.2−6.2
Bio Capabilities Index (SecureBio)Indexes171162−6.2
Eval Connections One-Shot with TrapsGames86.080.0−6.0
FACTS Benchmark SuiteGeneral QA65.371.2+5.9
CharXiv Extended (Mercor held-out)Multimodal84.779.1−5.6
GDP.pdf (Artificial Analysis, All-pass)Professional Work28.823.6−5.2
ChessBench EloGames13491265−5.0
ScBench V1 Clean RerunsScience55.450.4−5.0
Humanity's Sixth SenseMultimodal44.639.8−4.8
MindCube TinyMultimodal93.188.4−4.7
Excel Modeling BenchmarkFinance & Accounting75.971.3−4.6
GlobeBenchGeneral QA26.321.8−4.6
LMArena Agent Arena Bash ReliabilityAgents15.16.2−4.5
TextQuests Game ProgressGames57.352.9−4.4
Creative Writing v3 Rubric ScoreChat & Writing16.015.2−3.8
AA-AnalystAgent (pass^5)Professional Work56.360.0+3.8
MedCodeHealth & Medicine49.853.4+3.6
WCB (SecureBio)Science54.558.0+3.5
SAGEMath45.849.2+3.4
AIME 2025Math100.096.7−3.3
RedlineBenchLegal42.045.3+3.3
ChessBench CoherenceGames80.183.2+3.1
Terminal-Bench 2.1 Extended (Mercor held-out)Code47.544.4−3.1
GDPval (Mercor 213-task rubric score)Professional Work88.385.2−3.1
ARC-AGI-1 Semi-PrivateGeneral QA98.595.5−3.0
SimpleQA Verified (Epoch anti-abstention)General QA72.269.2−3.0
LMArena Text Arena EloChat & Writing15061489−2.9
MMLU ProGeneral QA93.490.6−2.8
OTIS Mock AIME 2024-2025Math100.097.2−2.8
ERQAMultimodal75.177.7+2.6
HardGeoBench <=10miMultimodal43.846.3+2.5
Terminal-Bench 2.1Code83.585.8+2.3
MedXpertQA MM (200-question set)Health & Medicine85.783.5−2.2
DTBenchGeneral QA98.996.8−2.1
BuseyBench SVGMultimodal6.86.6−2.1
Roboflow Vision Evals (October 2026, five tasks)Multimodal82.080.1−1.9
APEX-Agents 1.1Professional Work81.379.4−1.9
SpeciEval (Refusal-Aware Retries, Historical)Safety87.289.0+1.8
SpeciEval (Pooled English Runs)Safety87.289.0+1.8
MMMU ProMultimodal86.684.9−1.7
LVBenchMultimodal83.785.4+1.7
AA-LCR v1.1Search84.783.0−1.7
MMLU-Bio (SecureBio)Science92.791.4−1.3
HardGeoBench Country AccuracyMultimodal68.867.5−1.2
APEX-1 (Mercor, mean score)Professional Work65.867.0+1.2
SM BenchSafety81.880.6−1.1
CharXiv (descriptive and reasoning combined)Multimodal96.495.3−1.1
Chess Puzzles (task version 2.0)Games67.066.0−1.0
TextQuests Average Cumulative HarmSafety24.224.4−0.7
ForecastBench Preliminary Dataset Brier IndexForecasting62.261.7−0.5
Roboflow Vision Evals (September 2026 protocol)Multimodal85.585.2−0.3
ChessBench AccuracyGames79.379.2−0.1
IntPhys 2Multimodal68.368.20.0
LMArena Agent Arena Tool HallucinationAgents0.70.70.0
AHK-EvalCode91.791.70.0
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal0.00.00.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal0.00.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.00.00.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal20.020.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
ObviousBenchGeneral QA100.0100.00.0

Claude Opus 5.5vsGemini 3.7 Flashnormalised score marginsData Nov 15, 2026Data: Vals AI, Mercor, Arena and 52 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…