Atlas

Compare Models

Claude Sonnet 5.5 vs Gemini 3.8 Flash

Claude Sonnet 5.5 leads by 5.9 capability points (≈95% interval +3.5 to +8.4): a clear lead over Gemini 3.8 Flash · P(higher) ≈ 100%

Wins 189 of 234 shared benchmarks (43 lost, 2 tied)

Capability vs. relative cost
Claude Sonnet 5.5Gemini 3.8 FlashClosed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Claude Sonnet 5.5vsGemini 3.8 Flashcapability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins189 – 43 · 234 shared, 2 tied

Comparing matching setups only (24 benchmarks excluded) · suites counted once
← Claude Sonnet 5.5Gemini 3.8 Flash →

Claude Sonnet 5.5vsGemini 3.8 Flashwins by categoryData Nov 15, 2026Data: Artificial Analysis, Vals AI, Mercor and 66 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is Gemini 3.8 Flash minus Claude Sonnet 5.5.

AA-PersonalityBench Transparency indexChat & Writing10023.8−76.9
FrontierMath Tier 4 (v2) - PrivateMath80.522.0−58.5
Medical Long Context Reasoning (MLCR-AA)Health & Medicine75.021.7−53.3
ProofBench v1.1Math100.048.0−52.0
Olam Social Poker EloGames15261479−51.1
Terminal-Bench 4.0Code61.813.1−48.7
Code Migration v1.1Code86.938.6−48.2
FrontierML ($200 wallet)Agents50.45.7−44.7
AA-PersonalityBench Slop indexChat & Writing27.771.4+44.2
AA-PersonalityBench Imagination indexChat & Writing84.841.0−44.2
Code2Video-AA EloCode1443899−44.2
Furniture Assembly BenchmarkMultimodal75.031.7−43.3
FrontierSWE V2Code61.919.6−42.3
Short Form Bench v1.0Games23551940−41.1
ChessBench EloGames8571527+39.9
HoneyBench v0.2 (hack rate)Safety41.680.5−38.9
Terminal-Bench Science 0.1 (Vals)Science47.18.6−38.6
Vending-Bench 2Games111655094−36.8
Concept Synth INDUCTION Challenge64 Round 1General QA50.014.1−35.9
Concept Synth INDUCTION Challenge100 Round 1General QA44.09.0−35.0
ChessBenchGames39.373.3+34.0
AA-3D v1.0 EloCode1204974−33.3
MetagenomicsBench V1Science60.026.7−33.3
MetagenomicsBench V0Science60.026.7−33.3
Code MigrationCode69.836.5−33.3
SAFE-Teen 106 worst branch: Urgent Safety (no prompt)Safety23.256.4−33.2
Arena Image-to-WebDevCode17441594−32.8
Bug Hunt BenchCode48.917.1−31.7
MatFlowBench Pass@1Science50.018.9−31.1
APEX-SWECode66.436.4−30.0
sudo L7Code41.711.7−30.0
ChessBench CoherenceGames64.694.6+30.0
DAYJOB: AccountingFinance & Accounting35.26.0−29.2
GeneBench-Pro Public Release SubsetScience39.011.0−28.0
GameBench 2Games91.464.0−27.4
AA-PersonalityBench Conviction indexChat & Writing58.184.8+26.9
Artificial Analysis Coding Agent Index v1.5Indexes68.441.9−26.5
SAFE-Teen 106: Health Advice (no prompt)Safety33.259.6−26.4
ScBench V1 Clean RerunsScience60.334.0−26.3
IOI (Vals v2)Code83.156.9−26.1
SAFE-Teen 106 worst branch: Health Advice (no prompt)Safety42.969.0−26.1
LMArena Agent Arena Net Improvement (Steering Burden)Agents16.02.3−25.8
SAFE-Teen 106 worst branch: Relationships (no prompt)Safety37.062.6−25.7
LMArena Code Arena WebDevCode17731583−24.9
Decision-Bench v1.1Professional Work51.226.9−24.3
Terminal-Bench-Science 0.1Science34.310.0−24.3
Terminal-Bench 4.0 (Artificial Analysis)Code43.919.7−24.2
Humanity's Last Exam DiamondGeneral QA53.329.1−24.2
Artificial Analysis Intelligence Index v5.0.1Indexes54.530.8−23.7
Artificial Analysis Intelligence Index v5.0Indexes54.230.5−23.7
AA-Briefcase v1.1 EloAgents16341204−23.6
SAFE-Teen 106: Relationships (no prompt)Safety16.940.1−23.2
SimpleQA Verified (Epoch anti-abstention)General QA46.569.7+23.2
SAFE-Teen 106 worst branch: Urgent Safety (under-18 prompt)Safety11.834.8−23.0
TxBench-Antibody-Development V1 (60 evaluations)Science36.715.0−21.7
VibeVideoBench (overall quality, no video = 0)Code3.92.8−21.6
HANDBOOK.md Agents v1.1Professional Work47.726.5−21.2
DAYJOB: FinanceFinance & Accounting24.02.8−21.2
ChartographyMultimodal61.640.9−20.7
SAFE-Teen 106: Health Advice (under-18 prompt)Safety16.937.5−20.6
FrontierMath Tiers 1-3 (v2) - PrivateMath88.868.4−20.4
VariantBench V1 Clean RerunsScience51.731.4−20.3
RiemannBench (November 2026 rescoring)Math76.856.8−20.0
ViBench 2.0Code19.10.0−19.1
AA-LCR Extended (Mercor held-out)Search73.855.6−18.2
Mystery Game PuzzlesGames65.047.0−18.0
RuneBenchGames4.86.0+17.6
SWE-bench Multimodal (Vals v1, 480 Tasks)Code58.340.8−17.5
SAFE-Teen 106 worst branch: Health Advice (under-18 prompt)Safety30.848.1−17.3
PLE (Physical-Engineering's Last Exam)Professional Work45.228.2−17.0
Extended NYT Connections 940 (Quadratic v1)Games80.597.4+16.9
Physical-Engineering's Last Exam (hidden test)Professional Work45.128.2−16.9
NoCoT-Bench NCRI 15.2General QA95.1126+16.9
TxBench Oligo Discovery V1 (113 evaluations)Science51.935.1−16.8
SAFE-Teen 106: Urgent Safety (no prompt)Safety14.731.5−16.7
MKA (Artificial Analysis)Professional Work52.035.3−16.7
SAGEMath51.835.1−16.7
SWE-bench VerifiedCode96.780.6−16.1
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine29.313.3−16.0
BioSecBench-Function V1 Clean RerunsScience51.736.0−15.7
Conceptual Reasoning IndexIndexes77.165.7−15.7
Bio Capabilities Index (SecureBio)Indexes175153−15.5
LMCAGeneral QA68.352.9−15.4
LiveBench 2.0Agents82.567.3−15.1
Artificial Analysis Intelligence Index v4.3.2Indexes56.040.9−15.1
APEX-1 (Mercor, mean score)Professional Work53.968.8+14.9
HPCT (SecureBio)Science77.262.4−14.8
LLM Chess EloGames11951546+14.4
DAYJOB: HealthcareHealth & Medicine15.20.8−14.4
Vibe Code Bench v1.1Code92.478.7−13.7
HealthBench (length-adjusted, Artificial Analysis)Health & Medicine62.148.4−13.7
MaxxBenchCode66.953.4−13.5
AA-PersonalityBench Warmth indexChat & Writing73.360.0−13.5
SpatialBench V1 Clean RerunsScience72.859.4−13.3
CritPt (avg@5)Science31.418.3−13.1
LMArena Agent Arena Praise vs ComplaintAgents30.54.9−12.8
MysteryMechanismScience49.136.5−12.6
WeirdML v3Science19.97.4−12.6
Humanity's Last ExamGeneral QA56.944.5−12.4
EnterpriseBench: CoreCraftProfessional Work70.858.5−12.3
Eval Connections One-Shot with TrapsGames76.088.0+12.0
TaxCalcBench TY25Finance & Accounting18.06.0−12.0
SAFE-Teen 106: Urgent Safety (under-18 prompt)Safety2.914.9−12.0
HealthBench ProfessionalHealth & Medicine70.859.0−11.8
MineBench — Bradley-Terry RatingGames19981806−11.6
VIALSMultimodal41.029.4−11.6
AA-PersonalityBench Organization indexChat & Writing84.873.3−11.5
Hermes BenchAgents73.562.2−11.3
AA-AnalystAgent (pass^5)Professional Work57.546.3−11.2
Visa Agentic Commerce BenchmarkAgents64.453.3−11.1
FrontierCode 1.1 Extended - ScoreCode64.453.5−11.0
AutomationBench-AA 1.0.6Agents71.860.9−10.9
GDP.xlsxProfessional Work29.118.3−10.8
APEX-Agents 1.1Professional Work83.772.9−10.8
Artificial Analysis Progress Index v1Indexes351320−10.6
GDPval Extended (Mercor held-out rubric score)Professional Work70.960.4−10.5
ScBench-Long V1 Clean RerunsScience30.319.9−10.4
AML BenchFinance & Accounting67.557.1−10.4
FrontierCode 1.1 Extended - Pass RateCode69.959.6−10.3
SWE Atlas v1.1 - OverallCode67.557.3−10.2
BioSecBench-RespiratoryVirus-Surveillance V1 (110 evaluations)Science17.37.3−10.0
DAYJOB: TaxFinance & Accounting10.80.8−10.0
Veracode GenAI Code Security (Fall 2026)Cybersecurity72.062.0−10.0
WellnessBenchSafety57.047.0−10.0
CVDP (517-problem set)Code60.650.7−9.9
MBCT (SecureBio)Science58.848.9−9.9
SWE Atlas - Codebase QnACode50.340.6−9.7
TxBench Preclinical Pharmacology V1 Clean RerunsScience64.354.7−9.7
MissionReportProfessional Work63.854.3−9.5
ALE-BenchCode24482182−9.4
WCB (SecureBio)Science66.557.1−9.4
NoCoT-Bench NCKI (kspine v3)General QA107121+9.4
TreasuryQA BenchProfessional Work41.951.0+9.1
LMArena Agent Arena Confirmed SuccessAgents19.51.3−9.1
EpiBench V1 Clean RerunsScience30.221.4−8.8
GBENCH One-shot Coding GScoreGames67.458.6−8.7
HealthBenchHealth & Medicine65.957.1−8.7
Financial Coaching BenchFinance & Accounting74.966.3−8.7
Design Arena Overall Code CategoriesCode13491299−8.3
GDP.pdfProfessional Work31.423.2−8.2
LMArena Agent Arena Steering BurdenAgents14.5-1.3−7.9
Access to Justice BenchmarkLegal74.967.1−7.9
LMArena Agent Arena Bash RecoveryAgents15.0-0.7−7.9
Vision Arena OverallMultimodal12631293+7.8
ACCoRDGeneral QA85.778.2−7.5
Harvey's Legal Agent BenchmarkLegal2.910.0+7.1
Qodo AI Code Review Benchmark (125 PRs)Code54.047.0−7.0
VCT v2 (SecureBio)Science52.245.2−7.0
FrontierCode 2.0 (Artificial Analysis)Code25.718.7−7.0
SAFE-Teen 106 worst branch: Relationships (under-18 prompt)Safety7.314.0−6.7
MedScribeHealth & Medicine91.184.5−6.6
SimpleBenchGeneral QA75.982.4+6.5
SpatialBench-Long V1 Clean RerunsScience34.928.6−6.4
HardGeoBench <=100miMultimodal53.860.0+6.3
HardGeoBench <=10miMultimodal40.046.3+6.3
Epoch Capabilities Index (ECI)Indexes166159−6.2
FrontierCS Algorithmic v1.1Code53.947.8−6.1
micro1 PharmaBenchHealth & Medicine11.45.6−5.8
AA-PersonalityBench Expressiveness indexChat & Writing73.379.1+5.8
APEX-Accounting 1.1Finance & Accounting64.558.8−5.7
ChessBench AccuracyGames75.881.1+5.7
TxBench Antibody Discovery V1 (100 evaluations)Science54.749.0−5.7
VCT (SecureBio)Science52.546.9−5.6
Immigration BenchLegal69.063.4−5.6
ProgramBenchCode6.51.0−5.5
CharXiv Extended (Mercor held-out)Multimodal84.278.9−5.3
ARC-AGI-2General QA83.989.2+5.3
τ³-Banking (Mercor run, bm25_grep retrieval)Agents54.048.8−5.2
MedXpertQA MM Extended (Mercor held-out)Health & Medicine55.660.8+5.2
SWE-bench MultilingualCode90.985.8−5.1
HardGeoBench Country AccuracyMultimodal62.567.5+5.0
SM BenchSafety85.480.4−5.0
Toloka Arena (v3)Agents64.059.0−5.0
CritPt v2 (agentic, avg@5)Science70.065.1−4.9
GDPval-AA v2.1Professional Work15511442−4.8
MedCodeHealth & Medicine52.948.1−4.8
HardGeoBench Avg DistanceMultimodal689843−4.8
SciCode 1.0.1 (Artificial Analysis)Code61.056.6−4.4
LMArena Text Arena EloChat & Writing14721497+4.4
AA-LCR v1.1Search79.784.0+4.3
LAB-Bench LitQA2 (SecureBio)Science55.851.5−4.3
LAB-Bench ProtocolQA (SecureBio)Science88.384.0−4.3
ComplexConstraints (November 2026 rescoring)Chat & Writing67.263.2−4.0
MedXpertQA MM (200-question set)Health & Medicine80.784.7+4.0
AA-PersonalityBench Informativeness indexChat & Writing84.881.0−3.8
DistressBenchSafety74.670.7−3.8
Excel Modeling BenchmarkFinance & Accounting75.772.2−3.5
BrowseComp (Mercor 130-question subset)Search86.683.1−3.5
AIME 2025Math100.096.7−3.3
SWE-bench Verified Extended (Mercor held-out)Code57.354.0−3.3
Ramp Accounting BenchFinance & Accounting42.545.7+3.3
WMDP-Bio (SecureBio)Science92.689.4−3.2
Clipping Bench v1.0Games886596−3.1
Excel Modeling Benchmark v1.1Finance & Accounting75.972.9−3.0
DualEntry Accounting AI BenchmarkFinance & Accounting80.283.2+3.0
GDPval (Mercor 213-task rubric score)Professional Work86.583.5−3.0
LAB-Bench CloningScenarios (SecureBio)Science97.0100.0+3.0
FrontierMath ErdősMath2.90.0−2.9
Roboflow Vision Evals (October 2026, five tasks)Multimodal77.679.8+2.2
Quant-Arena (iterative total score)Finance & Accounting40.943.0+2.1
SAFE-Teen 106: Relationships (under-18 prompt)Safety1.73.7−2.1
ARC-AGI-1 Semi-PrivateGeneral QA96.598.5+2.0
Cresta-BenchAgents78.076.1−1.9
Public Benefits Bench v1.1Professional Work67.265.3−1.9
SWE-TogetherCode62.464.2+1.8
HLAB-Vals-1.1Legal10.412.1+1.7
DTBenchGeneral QA97.395.7−1.6
LMArena Agent Arena Bash ReliabilityAgents12.79.9−1.4
AA-Omniscience IndexIndexes32.329.6−1.4
Roboflow Vision Evals (September 2026 protocol)Multimodal83.885.1+1.3
MMMU ProMultimodal85.884.5−1.3
SpeciEval (Pooled English Runs)Safety91.089.9−1.1
OTIS Mock AIME 2024-2025Math100.098.9−1.1
SpeciEval (Refusal-Aware Retries, Historical)Safety91.089.9−1.1
Terminal-Bench 2.1 Extended (Mercor held-out)Code45.544.4−1.1
DAXBench v2Code86.287.2+1.1
Chess Puzzles (task version 2.0)Games59.058.0−1.0
BuseyBench SVGMultimodal6.56.6+1.0
RedlineBenchLegal40.341.3+1.0
SWE-Bench Pro V2 — FullCode95.894.9−0.9
Blueprint-Bench 2Multimodal0.40.4−0.7
BioSimBench 1.0Science16.015.3−0.7
CharXiv (descriptive and reasoning combined)Multimodal96.195.4−0.7
ForecastBench Preliminary Dataset Brier IndexForecasting61.160.5−0.6
DeepSWE v1.1Code70.571.0+0.5
ARC-AGI-3 (Semi-Private)General QA35.535.0−0.5
Terminal-Bench 2.1Code87.387.6+0.3
MMLU ProGeneral QA91.090.7−0.3
Prophet Arena Arena Score (Agentic, OpenCode)Forecasting85.786.0+0.3
LMArena Agent Arena Tool HallucinationAgents0.90.4−0.2
SAGE v1.1Math88.088.2+0.2
Prophet Arena Arena Score (Fixed Context)Forecasting85.385.30.0
Dual-Arm Sim (task completion)Agents0.00.00.0
ObviousBenchGeneral QA100.0100.00.0

Claude Sonnet 5.5vsGemini 3.8 Flashnormalised score marginsData Nov 15, 2026Data: Artificial Analysis, Vals AI, Mercor and 66 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…