Atlas

Compare Models

Claude Sonnet 5.5 vs Gemini 3.7 Flash

Claude Sonnet 5.5 leads by 7.2 capability points (≈95% interval +4.9 to +9.5): a clear lead over Gemini 3.7 Flash · P(higher) ≈ 100%

Wins 125 of 177 shared benchmarks (35 lost, 17 tied)

Capability vs. relative cost
Claude Sonnet 5.5Gemini 3.7 FlashClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Sonnet 5.5vsGemini 3.7 Flashcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins125 – 35 · 177 shared, 17 tied

Comparing matching setups only (9 benchmarks excluded) · suites counted once
← Claude Sonnet 5.5Gemini 3.7 Flash →

Claude Sonnet 5.5vsGemini 3.7 Flashwins by categoryData Nov 15, 2026Data: Vals AI, Mercor, Arena and 39 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Gemini 3.7 Flash minus Claude Sonnet 5.5.

Olam Social Poker EloGames15261468−63.0
Medical Long Context Reasoning (MLCR-AA)Health & Medicine75.015.0−60.0
Terminal-Bench 4.0Code61.86.1−55.8
Furniture Assembly BenchmarkMultimodal75.026.7−48.3
FrontierMath Tier 4 (v2) - PrivateMath80.536.6−43.9
ProofBench v1.1Math100.058.0−42.0
FrontierSWE V2Code61.920.3−41.6
Harvey's Legal Agent Benchmark - Energy Natural ResourcesLegal0.040.0+40.0
Terminal-Bench Science 0.1 (Vals)Science47.17.1−40.0
Code MigrationCode69.834.8−35.0
LMArena Agent Arena Net Improvement (Steering Burden)Agents16.0-2.2−34.3
Concept Synth INDUCTION Challenge100 Round 1General QA44.011.0−33.0
Concept Synth INDUCTION Challenge64 Round 1General QA50.017.2−32.8
Arena Code WebDev — ReactCode18221584−31.9
Arena Image-to-WebDevCode17441600−31.3
Arena Code WebDev — Brand & MarketingCode17611553−30.3
Terminal-Bench 4.0 (Artificial Analysis)Code43.913.6−30.3
Arena Code WebDev — Reference-Based DesignCode18241608−28.5
AA-Briefcase v1.1 EloAgents16341115−28.5
MetagenomicsBench V1Science60.031.7−28.3
MetagenomicsBench V0Science60.031.7−28.3
Mystery Game PuzzlesGames65.037.0−28.0
Bug Hunt BenchCode48.921.0−27.9
Arena Code WebDev — FrontendCode18081592−27.7
Arena Code WebDev — GamingCode18841638−27.6
SWE-bench Verified Extended (Mercor held-out)Code57.330.0−27.3
Arena Code WebDev — SimulationsCode18411618−25.3
GeneBench-Pro Public Release SubsetScience39.014.0−25.0
AA-LCR Extended (Mercor held-out)Search73.848.9−24.9
APEX-SWECode66.442.0−24.4
ChessBench EloGames8571265+24.3
SimpleQA Verified (Epoch anti-abstention)General QA46.569.2+22.7
Arena Code WebDev — HTMLCode17521593−22.3
Humanity's Last Exam DiamondGeneral QA53.331.2−22.1
Vibe Code Bench v1.1Code92.470.4−22.0
Arena Code WebDev — Content Creation ToolsCode17091574−21.4
ChartographyMultimodal61.640.4−21.2
DAYJOB: FinanceFinance & Accounting24.02.8−21.2
VariantBench V1 Clean RerunsScience51.731.4−20.3
ALE-BenchCode24481876−20.3
Harvey's Legal Agent Benchmark - Banking FinanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Capital MarketsLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Corporate GovernanceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Data Privacy CybersecurityLegal20.00.0−20.0
Harvey's Legal Agent Benchmark - Healthcare Life SciencesLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - InsuranceLegal0.020.0+20.0
Harvey's Legal Agent Benchmark - Trusts Estates Private ClientLegal20.00.0−20.0
Harvey HLAB — Criteria Pass RateLegal70.088.6+18.6
ChessBench CoherenceGames64.683.2+18.6
BioSecBench-Function V1 Clean RerunsScience51.733.7−18.1
LMCAGeneral QA68.350.4−17.9
RuneBenchGames4.86.0+17.6
Conceptual Reasoning IndexIndexes77.164.6−17.2
FrontierMath Tiers 1-3 (v2) - PrivateMath88.871.6−17.2
CritPt (avg@5)Science31.414.3−17.1
NoCoT-Bench NCRI 15.2General QA95.1126+17.0
Arena Code WebDev — FullstackCode16941629−17.0
Artificial Analysis Intelligence Index v4.3.2Indexes56.039.1−16.9
LMArena Agent Arena Praise vs ComplaintAgents30.5-2.9−16.7
SWE-bench VerifiedCode96.781.0−15.7
LiveBench 2.0Agents82.567.2−15.3
IOI (Vals v2)Code83.167.8−15.2
DAYJOB: HealthcareHealth & Medicine15.20.0−15.2
EnterpriseBench: CoreCraftProfessional Work70.855.9−14.9
GBENCH One-shot Coding GScoreGames67.452.6−14.8
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine29.314.7−14.7
VIALSMultimodal41.026.5−14.5
TxBench Oligo Discovery V1 (113 evaluations)Science51.937.8−14.2
MineBench — Bradley-Terry RatingGames19981765−14.1
SpatialBench V1 Clean RerunsScience72.859.1−13.6
GDPval Extended (Mercor held-out rubric score)Professional Work70.957.8−13.1
APEX-1 (Mercor, mean score)Professional Work53.967.0+13.1
HPCT (SecureBio)Science77.264.7−12.5
Veracode GenAI Code Security (Fall 2026)Cybersecurity72.060.0−12.0
Artificial Analysis Progress Index v1Indexes351317−11.6
ScBench-Long V1 Clean RerunsScience30.318.8−11.5
LMArena Agent Arena Confirmed SuccessAgents19.5-3.5−11.5
LMArena Agent Arena Steering BurdenAgents14.5-7.3−10.9
LMArena Agent Arena Bash RecoveryAgents15.0-5.4−10.2
ScBench V1 Clean RerunsScience60.350.4−9.9
AutomationBench-AA 1.0.6Agents71.862.0−9.7
NoCoT-Bench NCKI (kspine v3)General QA107121+9.7
Bio Capabilities Index (SecureBio)Indexes175162−9.2
TxBench Preclinical Pharmacology V1 Clean RerunsScience64.355.3−9.0
ACCoRDGeneral QA85.777.0−8.7
WCB (SecureBio)Science66.558.0−8.5
SWE-bench MultilingualCode90.982.6−8.3
DeepResearch Bench II (Mercor 130-task set)Search51.743.4−8.3
EpiBench V1 Clean RerunsScience30.222.0−8.2
FrontierCode 1.1 Extended - ScoreCode64.456.3−8.2
Vision Arena OverallMultimodal12631294+8.0
Arena Code WebDev — Consumer ProductCode16361574−7.8
GDP.pdfProfessional Work31.423.8−7.6
FrontierCode 1.1 Extended - Pass RateCode69.962.3−7.5
Design Arena Overall Code CategoriesCode13491304−7.5
BrowseComp (Mercor 130-question subset)Search86.679.2−7.4
APEX-Accounting 1.1Finance & Accounting64.557.2−7.3
MedScribeHealth & Medicine91.183.9−7.2
MedXpertQA MM Extended (Mercor held-out)Health & Medicine55.662.7+7.1
Chess Puzzles (task version 2.0)Games59.066.0+7.0
ProgramBenchCode6.50.0−6.5
GDPval-AA v2.1Professional Work15511404−6.5
Epoch Capabilities Index (ECI)Indexes166158−6.4
HardGeoBench Avg DistanceMultimodal689895−6.4
HardGeoBench <=10miMultimodal40.046.3+6.3
Toloka Arena (v3)Agents64.058.0−6.0
GPQA-Bio (SecureBio)Science87.681.6−6.0
FrontierCS Algorithmic v1.1Code53.948.0−5.9
DistressBenchSafety74.669.0−5.6
SpatialBench-Long V1 Clean RerunsScience34.929.8−5.2
CharXiv Extended (Mercor held-out)Multimodal84.279.1−5.1
Arena Code WebDev — Data & AnalyticsCode16101577−5.0
DeepSWE v1.1Code70.565.5−5.0
HardGeoBench Country AccuracyMultimodal62.567.5+5.0
RedlineBenchLegal40.345.3+5.0
SM BenchSafety85.480.6−4.8
Excel Modeling BenchmarkFinance & Accounting75.771.3−4.4
APEX-Agents 1.1Professional Work83.779.4−4.3
Eval Connections One-Shot with TrapsGames76.080.0+4.0
TaxCalcBench TY25Finance & Accounting18.014.0−4.0
TxBench Antibody Discovery V1 (100 evaluations)Science54.750.7−4.0
MBCT (SecureBio)Science58.855.0−3.8
ChessBench AccuracyGames75.879.2+3.6
VCT v2 (SecureBio)Science52.248.6−3.6
LLM Chess EloGames11951107−3.6
AA-LCR v1.1Search79.783.0+3.3
AIME 2025Math100.096.7−3.3
LMArena Agent Arena Bash ReliabilityAgents12.76.2−3.3
VCT (SecureBio)Science52.549.3−3.2
LMArena Text Arena EloChat & Writing14721489+3.0
AA-Omniscience IndexIndexes32.326.5−2.9
MedXpertQA MM (200-question set)Health & Medicine80.783.5+2.8
OTIS Mock AIME 2024-2025Math100.097.2−2.8
SAGEMath51.849.2−2.5
AA-AnalystAgent (pass^5)Professional Work57.560.0+2.5
HardGeoBench <=100miMultimodal53.856.3+2.5
Roboflow Vision Evals (October 2026, five tasks)Multimodal77.680.1+2.5
LAB-Bench CloningScenarios (SecureBio)Science97.099.4+2.4
SpeciEval (Pooled English Runs)Safety91.089.0−1.9
SpeciEval (Refusal-Aware Retries, Historical)Safety91.089.0−1.9
LAB-Bench LitQA2 (SecureBio)Science55.854.2−1.6
Terminal-Bench 2.1Code87.385.8−1.5
LAB-Bench ProtocolQA (SecureBio)Science88.386.8−1.5
Roboflow Vision Evals (September 2026 protocol)Multimodal83.885.2+1.4
GDPval (Mercor 213-task rubric score)Professional Work86.585.2−1.3
SciCode 1.0.1 (Artificial Analysis)Code61.059.8−1.2
Terminal-Bench 2.1 Extended (Mercor held-out)Code45.544.4−1.1
BuseyBench SVGMultimodal6.56.6+1.1
ARC-AGI-1 Semi-PrivateGeneral QA96.595.5−1.0
MMMU ProMultimodal85.884.9−0.9
CharXiv (descriptive and reasoning combined)Multimodal96.195.3−0.8
ARC-AGI-2General QA83.984.6+0.7
ForecastBench Preliminary Dataset Brier IndexForecasting61.161.7+0.6
DTBenchGeneral QA97.396.8−0.5
Humanity's Last ExamGeneral QA46.145.6−0.5
MedCodeHealth & Medicine52.953.4+0.5
MMLU ProGeneral QA91.090.6−0.4
MMLU-Bio (SecureBio)Science91.091.4+0.4
Ramp Accounting BenchFinance & Accounting42.542.3−0.2
LMArena Agent Arena Tool HallucinationAgents0.90.7−0.1
Harvey's Legal Agent Benchmark - Antitrust CompetitionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Arbitration International Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Bankruptcy RestructuringLegal0.00.00.0
Harvey's Legal Agent Benchmark - Corporate M&ALegal0.00.00.0
Harvey's Legal Agent Benchmark - Emerging Companies Venture CapitalLegal0.00.00.0
Harvey's Legal Agent Benchmark - Employment LaborLegal0.00.00.0
Harvey's Legal Agent Benchmark - Environmental ESGLegal0.00.00.0
Harvey's Legal Agent Benchmark - Funds Asset ManagementLegal0.00.00.0
Harvey's Legal Agent Benchmark - ImmigrationLegal0.00.00.0
Harvey's Legal Agent Benchmark - Intellectual PropertyLegal0.00.00.0
Harvey's Legal Agent Benchmark - International Trade SanctionsLegal0.00.00.0
Harvey's Legal Agent Benchmark - Litigation Dispute ResolutionLegal0.00.00.0
Harvey's Legal Agent Benchmark - Real EstateLegal0.00.00.0
Harvey's Legal Agent Benchmark - Structured Finance SecuritizationLegal0.00.00.0
Harvey's Legal Agent Benchmark - TaxLegal0.00.00.0
Harvey's Legal Agent Benchmark - White Collar Defense InvestigationsLegal0.00.00.0
ObviousBenchGeneral QA100.0100.00.0

Claude Sonnet 5.5vsGemini 3.7 Flashnormalised score marginsData Nov 15, 2026Data: Vals AI, Mercor, Arena and 39 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…