Atlas

Compare Models

Opus 4.8 vs GPT-5.5

GPT-5.5 leads by 0.3 capability points (≈95% interval −1.6 to +2.2): statistically tied · P(higher) ≈ 62%

Wins 184 of 335 shared benchmarks (144 lost, 7 tied)

Capability vs. relative cost
Opus 4.8GPT-5.5Closed weightsOpen weights

Each line runs through one model’s priced reasoning settings, cheapest to most expensive; grey points are other models’ priced settings in the same range. Results whose reasoning effort the source didn’t report are left out wherever a model has named effort levels. Capability and relative cost are the published estimates (relative cost is cost per task as a multiple of GPT-5 (high)); hover a setting, or use the arrow keys, to set it beside the other model’s same setting.

Opus 4.8vsGPT-5.5capability vs. relative costData Nov 15, 2026Village LLM Atlasatlas.village.dev

Benchmark wins144 – 184 · 335 shared, 7 tied

Comparing matching setups only (149 benchmarks excluded) · suites counted once
← Opus 4.8GPT-5.5 →

Opus 4.8vsGPT-5.5wins by categoryData Nov 15, 2026Data: Vals AI, benchmarks.bio, Mercor and 94 othersVillage LLM Atlasatlas.village.dev

Margins

Each benchmark scaled 0–100% (its declared range, else the field’s range); Δ is GPT-5.5 minus Opus 4.8.

Vending-Bench Arena: Round 8Games518810627+100.0
ExploitBench Safeguard Blocking (CAISI 10-task set)Safety36.0300−88.0
HPCT Refusal (CAIS)Safety85.029.0−56.0
Vending-Bench Arena: Round 9Games61968288+51.7
BullshitBench v2 LeaderboardSafety94.047.0−47.0
LLM Chess EloGames4501532+44.4
Clocktower BenchGames11951486+41.8
KernelBench Mega Normalized (H100)Code63.823.1−40.7
PrinzBenchLegal42.074.0+32.3
Opus Magnum CampaignGames22.853.9+31.1
BrokenArXiv — OverallMath33.063.9+30.8
SnakeBench Win RateGames60.090.0+30.0
BenchCAD - geometric overlapMultimodal27.355.8+28.5
Medical Long Context Reasoning (MLCR-AA)Health & Medicine45.617.8−27.8
VCT-Refusal (CAIS)Safety41.216.0−25.2
SWE-bench Verified Extended (Mercor held-out)Code63.339.3−24.0
SimpleQA VerifiedGeneral QA39.563.1+23.6
ChessBench EloGames9611349+23.1
GraphWalks BFS - 1MGeneral QA68.145.4−22.7
Blueprint-Bench 2Multimodal0.10.4+21.7
BioTIER RefusalSafety94.773.4−21.3
VoxelBench Text RatingGames16271925+21.2
PACT Negotiation Bilateral RatingGames15621607+21.1
ChessBench CoherenceGames79.7100.0+20.3
Mystery Game PuzzlesGames36.056.0+20.0
SWE-Marathon v1.0Code26.06.0−20.0
ProofBenchMath69.050.0−19.0
LLM Creative Story WritingChat & Writing0.82.4+18.9
LLM Position Bias First-Shown Pick RateChat & Writing60.769.9−18.4
LLM Position Bias First-Option LiftChat & Writing10.719.9−18.4
Drone-BenchAgents82.363.9−18.4
GBA Eval OverallGames70.953.2−17.7
YC-BenchGames17113891206045−17.6
LLM Position Bias Order Flip RateChat & Writing27.444.4−17.0
EnigmaEvalGeneral QA20.537.2+16.7
Visual Acuity 4K Threshold RowMultimodal10.012.0+16.7
MCPMark-VerifiedAgents76.492.9+16.5
FrontierMath Tier 4 (v2) - PrivateMath56.172.5+16.4
TasteVal (compute multiplier)Science0.80.5−16.3
Aikido Known-CVE Detection (Pass@3 Union)Cybersecurity69.284.6+15.4
A-FantasiaGeneral QA50.034.7+15.3
MindCube TinyMultimodal64.979.9+15.0
BioSecBench-Function V1 Clean RerunsScience43.128.3−14.8
LLM Debate BenchmarkChat & Writing16311528−14.7
GDPval-AA v1Professional Work18901769−14.5
DeepsecBenchCode6.921.2+14.3
DeepsecBench — Revalidated V1Code6.921.1+14.2
DeepsecBench — Revalidated V1 (233 issues)Code6.921.0+14.2
ComplexConstraintsChat & Writing35.649.5+13.9
SpeciEval (Pooled English Runs)Safety84.398.2+13.9
SpeciEvalSafety84.398.2+13.9
SpeciEval (Refusal-Aware Retries, Historical)Safety84.398.2+13.9
PCT Political Inconsistency (CAIS)Safety53.840.0+13.8
IFBench (Artificial Analysis avg@5)Chat & Writing62.275.9+13.6
Visual Math — OverallMath81.694.9+13.3
ARC-AGI-2General QA72.185.0+12.9
Vibe Code Bench v1.1Code82.769.8−12.9
Terminal-Bench 2.0Code70.082.7+12.7
DeepSWE v1.1Code51.864.4+12.6
VariantBenchScience42.129.7−12.4
MMMU-Pro Extended (Mercor held-out)Multimodal31.443.8+12.4
DeepSWE v1Code57.770.0+12.3
GraphWalks BFS - 256KGeneral QA85.973.7−12.2
OfficeQA ProProfessional Work66.254.1−12.1
TxBench Antibody Discovery V0Science45.633.6−12.1
VariantBench V1 Clean RerunsScience42.430.5−11.9
CRED v0.92Science85.096.5+11.5
AA-Briefcase EloAgents13541158−11.4
AURORA Multilingual Terminal-BenchCode49.160.3+11.2
BrowseComp Extended (Mercor held-out)Search30.019.0−11.0
Chess Puzzles (task version 2.0)Games52.063.0+11.0
ERQAMultimodal59.570.5+11.0
ArXivMath — OverallMath64.875.6+10.8
MRCR v2 (8-needle, Context Arena) - 128K pointwiseGeneral QA75.285.8+10.7
SWE-Bench Pro (Public Dataset)Code69.258.6−10.6
PMPP-HardCode62.352.2−10.1
JobBench MainProfessional Work48.438.3−10.1
ALE-BenchCode19131633−9.9
GraphWalks Parents - 256K F1General QA99.390.1−9.2
MedXpertQA MM Extended (Mercor held-out)Health & Medicine51.660.8+9.2
PointerbenchMultimodal68.959.9−9.0
SpatialViz-BenchMultimodal65.274.2+9.0
HieroglyphBenchMultimodal14.022.8+8.8
TextQuests Game ProgressGames40.148.9+8.8
Document ArenaProfessional Work14751484+8.7
LMArena Agent Arena Net ImprovementAgents6.94.2−8.7
PerceptionBenchMultimodal47.255.8+8.6
BioSecBench-RefusalSafety28.637.1+8.5
Roboflow Vision EvalsMultimodal68.176.4+8.3
ActiveVisionMultimodal2.410.6+8.2
GBENCH One-shot Coding GScoreGames54.062.2+8.2
BLXBenchCode69.777.9+8.2
BioSecBench-Function V0Science44.636.5−8.1
LiveBench 2.0Agents76.868.7−8.1
CadQueryEval (Corrected Geometry)Code68.076.0+8.0
RiemannBenchMath47.255.2+8.0
CadQueryEvalCode56.064.0+8.0
Apex ShortlistMath90.498.4+8.0
BuseyBench SVGMultimodal5.26.0+7.9
ExploitBench - release-reported exploit progressCybersecurity40.047.9+7.9
GDPval-AA v2Professional Work15291372−7.7
AdvancedIFChat & Writing73.781.3+7.6
SM BenchSafety72.965.4−7.5
AA-LCR Extended (Mercor held-out)Search63.270.5+7.3
OSWorld 2.0 500-step Partial ScoreAgents54.847.5−7.3
Public Benefits Bench v1.1Professional Work68.160.9−7.2
EQ-Bench 4Chat & Writing12681295+7.2
Terminal-Bench 4.0 (Artificial Analysis)Code21.714.6−7.1
SnakeBench Average ApplesGames15.116.5+7.0
WCB (SecureBio)Science66.759.7−7.0
CAISI Cyber Safeguards - No JailbreakSafety7.00.0+7.0
ZeroBench (Main questions, pass@5, with tools)Multimodal34.041.0+7.0
MCP Atlas April 2026 (all 1,000 tasks)Agents82.275.3−6.9
GSO Opt@1Code47.140.2−6.9
ChessBench AccuracyGames72.478.8+6.9
SWE Atlas Codebase QnA Extended (Mercor held-out)Code24.017.3−6.7
LisanBench Difficulty-WeightedGames26943316+6.7
τ³-bench averageAgents67.761.1−6.6
GPQA-Bio (SecureBio)Science80.987.4+6.5
SUPERChemScience60.366.8+6.5
APEX-SWECode47.340.8−6.5
SkillsBench without SkillsAgents43.749.9+6.2
Machiavelli Deception (CAIS)Safety83.781.7+6.2
Antidote: Everyday EditionSafety10301013−6.1
CharXiv Extended (Mercor held-out)Multimodal76.782.7+6.0
Humanity's Last ExamGeneral QA49.843.8−6.0
SWE-bench VerifiedCode88.682.6−6.0
EpiBenchScience39.045.0+6.0
SnakeBench Best ApplesGames28.026.0−5.9
OSWorld-VerifiedAgents83.477.5−5.9
Harvey's Legal Agent BenchmarkLegal9.63.8−5.8
TxBench Oligo Discovery V0Science39.433.6−5.8
EBR-bench v3Games28.634.3+5.7
APEX-1 (Mercor, mean score)Professional Work62.768.4+5.7
BankerToolBench v1.1Finance & Accounting20.715.0−5.7
MMMU ProMultimodal75.681.2+5.6
MATH-VisionMultimodal86.792.2+5.5
Agent Red Teaming k=100 (CAIS)Safety47.041.5+5.5
SnakeBench RatingGames35.437.4+5.4
ChessBench — Historical Output RepairGames24.029.3+5.3
Harvey LAB (Mercor run, 1,749 public tasks)Legal9.44.1−5.3
Terminal-Bench 2.1 Extended (Mercor held-out)Code32.738.0+5.3
FrontierMath Tiers 1-3 (v2) - PrivateMath80.085.3+5.3
GlobeBenchGeneral QA20.025.2+5.1
Ramp SWE-Bench (78 published tasks)Code78.283.3+5.1
ScBench-Long V1 Clean RerunsScience16.521.6+5.1
MineBench Conservative RatingGames16861772+5.1
Public Benefits Bench v1 - Web SearchProfessional Work58.953.9−5.1
AA-AnalystAgent (pass^5)Professional Work45.050.0+5.0
Big Finance Bench - OpenAI release scoreFinance & Accounting44.049.0+5.0
CAISI Cyber Safeguards - Best Public JailbreakSafety7.012.0−5.0
GDPval-AA v2 (normalized)Professional Work54.549.5−5.0
Ruby LLM Benchmarks — Program FixerCode72.677.6+4.9
Public Benefits Bench v1 - BothProfessional Work62.157.2−4.9
Excel Modeling BenchmarkFinance & Accounting69.464.5−4.8
DrugDiscoveryBenchScience46.851.6+4.8
Humanity's Last Exam DiamondGeneral QA23.127.9+4.8
IntPhys 2Multimodal55.159.9+4.8
ScBench-LongScience20.615.9−4.8
𝜏³-BankingAgents34.239.0+4.7
Long-Horizon Terminal-Bench 1.0 (mean reward)Code49.244.5−4.7
Diligence Stack Agent BenchFinance & Accounting77.382.0+4.7
Tampered MLScience76.571.8−4.7
Public Benefits Bench v1 - Multi TurnProfessional Work58.553.9−4.6
SciCode (main problems)Science16.912.3−4.6
PubMedQA (SecureBio)Health & Medicine75.980.5+4.6
GDPval-AA v2.1Professional Work14611357−4.6
SpatialBench-Long V0 VerifiedScience28.833.3+4.5
NoCoT-Bench NCKI (kspine v3)General QA117111−4.5
Tax Agent BenchFinance & Accounting64.960.5−4.5
DeepSearchQA (F1 score)Search92.287.8−4.4
RedlineBenchLegal39.744.1+4.4
GDPval Extended (Mercor held-out rubric score)Professional Work58.954.5−4.4
HiL-Bench (Human-in-Loop Benchmark)Agents35.339.7+4.3
MASKSafety94.490.1−4.3
AutomationBench (Overall)Agents17.212.9−4.3
ToolathlonAgents59.955.6−4.3
EBR-bench v2Games25.221.0−4.3
Bio Capabilities Index (SecureBio)Indexes164158−4.3
MedXpertQA MM (200-question set)Health & Medicine72.576.7+4.2
SimpleBenchGeneral QA64.869.0+4.2
WebArena-VerifiedAgents71.267.0−4.2
MedCodeHealth & Medicine53.249.1−4.1
AA-Omniscience IndexIndexes28.820.5−4.1
VCT (SecureBio)Science48.252.3+4.1
BabaIsBench Stage 0: The IntroGames96.0100.0+4.0
TaxCalcBench TY25Finance & Accounting20.024.0+4.0
TxBench-PPScience59.355.3−4.0
MRCR v2 (8-needle, Context Arena) - 128K AUCGeneral QA87.891.7+3.9
CursorBench 3.2Code62.358.4−3.9
TeachArenaProfessional Work80.376.4−3.9
UI4A-BenchCode75.972.1−3.8
Deep Research BenchSearch0.50.5+3.8
HPCT (SecureBio)Science63.166.8+3.7
OEIS Open ($50)Math29.926.2−3.7
MineBench — Bradley-Terry RatingGames17861846+3.6
CorpFin v2 - Shared Max ContextFinance & Accounting66.069.6+3.6
CorpFin v2 (Vals Index component)Finance & Accounting66.069.6+3.6
CorpFin v2 (Vals Multimodal Index component)Finance & Accounting66.069.6+3.6
CharXiv ReasoningMultimodal80.584.1+3.6
GPQA DiamondGeneral QA91.995.5+3.6
τ²-bench (Telecom)Agents94.498.0+3.6
Time Horizon Index: KSPAgents11.88.3−3.5
Vals Index v2Indexes60.957.4−3.5
Artificial Analysis Intelligence Index v4.3.2Indexes41.838.4−3.4
SnakeBench Total ApplesGames226181−3.4
Extended NYT Connections 940-Puzzle SetGames94.197.5+3.4
Public Benefits Bench v1 - NeitherProfessional Work53.149.7−3.4
Legal Research BenchLegal43.840.4−3.4
Artificial Analysis Intelligence Index v4.3Indexes42.038.6−3.4
TxBench Oligo Discovery V1 Clean RerunsScience35.331.9−3.3
Video-MME (with subtitles)Multimodal86.089.3+3.3
Olam Social Poker EloGames15181521+3.3
SAGEMath54.851.5−3.3
LMCAGeneral QA57.554.3−3.2
LifeSciBench - release-reported version unspecifiedScience53.650.4−3.2
ACCoRDGeneral QA77.480.5+3.1
HMMT Feb 2026Math95.598.5+3.0
ScBench-Long V0 VerifiedScience25.822.7−3.0
WordleBenchGames94.097.0+3.0
SkillsBench with SkillsAgents59.262.2+3.0
LegalBenchLegal83.686.5+2.9
Vals Multimodal IndexIndexes70.968.1−2.8
SpatialBench-LongScience8.311.1+2.8
AHK-EvalCode100.097.2−2.8
Macaron ChatBenchChat & Writing52.855.5+2.7
EnterpriseOps-Gym-AAAgents44.046.6+2.7
GameBench 2Games62.559.9−2.5
EpiBench V1 Clean RerunsScience25.828.3+2.5
ARC-AGI-1 Semi-PrivateGeneral QA92.595.0+2.5
MMVUMultimodal79.281.7+2.5
Final-Answer Comps — OverallMath91.894.3+2.5
Vals IndexIndexes70.468.0−2.4
BabyVision with PythonMultimodal81.283.6+2.4
BabyVision with toolsMultimodal81.283.6+2.4
TxBench Preclinical Pharmacology V1 Clean RerunsScience57.755.3−2.3
Artificial Analysis Agentic IndexIndexes47.244.9−2.3
Terminal-Bench HardCode58.360.6+2.3
Terminal-Bench 2.1Code82.084.3+2.3
Finance Agent v2Finance & Accounting53.951.8−2.1
ForecastBench Baseline Overall Brier IndexForecasting58.160.2+2.1
Code MigrationCode47.345.2−2.1
LiveCodeBench (Vals Public) - MediumCode89.887.7−2.1
WeirdML (v2)Science82.984.9+2.0
ClawGym-BenchAgents80.582.5+2.0
GDP.pdfProfessional Work24.026.0+2.0
Toloka Arena (v1)Agents54.352.3−2.0
Macaron LivingBenchAgents63.861.9−1.9
GeneBench-ProScience10.112.0+1.9
FrontierSWE (Dominance)Code66.764.9−1.8
SpatialBench V1 Clean RerunsScience65.864.1−1.7
SciCode 1.0.1 (Artificial Analysis)Code54.456.1+1.7
AIME 2025Math98.3100.0+1.7
ForecastBench Preliminary Dataset Brier IndexForecasting61.663.3+1.7
InteractiveBench (task pass)Agents28.330.0+1.7
Furniture Assembly BenchmarkMultimodal42.544.2+1.7
OTIS Mock AIME 2024-2025Math98.3100.0+1.7
FrontierFinance v3.1Finance & Accounting45.043.5−1.6
USAMO 2026Math96.798.2+1.5
OmniDocBench v1.5Multimodal87.989.4+1.5
CharXiv (descriptive and reasoning combined)Multimodal93.294.6+1.4
LAB-Bench CloningScenarios (SecureBio)Science95.997.3+1.4
ATLAS-Education (StudentBench) Lesson PlanningProfessional Work0.90.9−1.4
LMArena Agent Arena SteerabilityAgents9.66.9−1.4
TextQuests Average Cumulative HarmSafety21.021.4−1.4
OncologyBench Strict (25 tasks, October 9 rubric)Health & Medicine16.014.7−1.3
Conceptual Reasoning IndexIndexes68.367.3−1.3
DECK-BenchProfessional Work66.968.2+1.3
ExploitGymCybersecurity13.815.1+1.3
CorpFin v2 - Exact PagesFinance & Accounting64.365.6+1.3
ATLAS-Education (StudentBench) Learning GainProfessional Work16.013.5−1.3
Realm Financial ReasoningFinance & Accounting43.144.3+1.2
SpatialBench-Long V1 Clean RerunsScience33.332.1−1.2
LisanBench Path LengthGames94639845+1.2
LMArena Agent Arena Bash RecoveryAgents7.29.5+1.2
SpatialBench V0 VerifiedScience66.465.2−1.2
MortgageTax (Vals all 1,258 documents)Finance & Accounting69.968.8−1.2
MedScribeHealth & Medicine85.886.9+1.1
VitaBench 2.0 - Rewrite (Agentic Memory)Agents46.347.4+1.1
LAB-Bench LitQA2 (SecureBio)Science48.749.8+1.1
MMLU ProGeneral QA90.088.9−1.1
ARC-AGI-3 (Semi-Private)General QA1.50.4−1.1
DTBenchGeneral QA94.996.0+1.1
SpatialBenchScience52.653.7+1.1
Apex (MathArena)Math81.380.2−1.0
scBenchScience56.857.8+1.0
EnterpriseBench: CoreCraftProfessional Work52.351.3−1.0
LAB-Bench ProtocolQA (SecureBio)Science76.977.9+1.0
Senior SWE-BenchCode30.529.5−1.0
NoCoT-Bench NCRI 15.2General QA102100−1.0
LiveBenchGeneral QA78.979.9+1.0
PG-LLM-50Science0.40.3−1.0
APEX-AgentsProfessional Work39.438.5−0.9
Hemingway-benchChat & Writing10381035−0.8
Design Arena Overall Code CategoriesCode12521257+0.8
SWE Atlas - Codebase QnACode44.445.2+0.8
DeepResearch Bench II (Mercor 130-task set)Search44.645.4+0.8
ForecastBench Tournament Overall Brier IndexForecasting66.267.0+0.8
MCP Atlas public (Kimi K3 100-turn evaluation)Agents83.682.8−0.8
SEA-HELMGeneral QA75.676.3+0.7
MBCT (SecureBio)Science56.457.1+0.7
VitaBenchProfessional Work56.555.8−0.7
ObviousBenchGeneral QA99.3100.0+0.7
Vision Arena OverallMultimodal12861283−0.7
Agents' Last Exam (ALE)Agents22.423.0+0.7
TaxEval v2Finance & Accounting75.675.0−0.7
Artificial Analysis Coding IndexIndexes74.374.9+0.6
Surface Evolver BenchAgents87.588.1+0.6
LiveCodeBench (Vals Public) - EasyCode99.198.4−0.6
VCT v2 (SecureBio)Science45.945.3−0.6
WorldVQA ForceAnswerMultimodal39.138.5−0.6
LLM Position Bias Decisive Pair CoverageChat & Writing92.793.3+0.6
Artificial Analysis Progress Index v1Indexes318320+0.6
ProgramBenchCode1.00.5−0.5
DiligenceBenchFinance & Accounting47.147.5+0.4
HANDBOOK.mdProfessional Work21.921.5−0.4
LLM Position Bias Rating BonusChat & Writing0.20.2+0.3
AutomationBench 1.0.6 (Zapier)Agents16.916.6−0.3
OpenAI management-consulting tasks - internal release scoreProfessional Work31.631.3−0.3
Bench to the Future 3 — Pooled ScoreForecasting0.10.1−0.3
TxBench Antibody Discovery V1 Clean RerunsScience30.229.9−0.2
CorpFin v2 - Max Fitting ContextFinance & Accounting69.870.0+0.2
APEX-AccountingFinance & Accounting48.048.2+0.2
BioSecBench-Surveillance V1 Clean RerunsScience46.947.1+0.2
Epoch Capabilities Index (ECI)Indexes160160+0.1
BrowseCompSearch84.384.4+0.1
GDPval (Mercor 213-task rubric score)Professional Work84.584.6+0.1
LMArena Text Arena EloChat & Writing14821481−0.1
A-Fantasia (Valid Attempts with Retries)Games34.734.70.0
AIME 2026Math100.0100.00.0
BioSecBench-SurveillanceScience50.250.20.0
CAISI Biology Safeguards - Best Public JailbreakSafety0.00.00.0
CAISI Biology Safeguards - No JailbreakSafety0.00.00.0
KernelBench Hard Normalized (H100)Code0.00.00.0
ScBench V1 Clean RerunsScience57.157.10.0

Opus 4.8vsGPT-5.5normalised score marginsData Nov 15, 2026Data: Vals AI, benchmarks.bio, Mercor and 94 othersVillage LLM Atlasatlas.village.dev

Win counts use each benchmark’s declared score direction. Represented suite slices and the derived Capability score are excluded from headline totals. Missing or incompatible setups are excluded, never counted as losses.
Loading Atlas data…