Benchmarks
357 sources · 1,263 models × 3,079 benchmarks · 162,127 scoresMedChemBench's numerical prediction component (Table S2 of the paper): on the 21 of 99 tasks with numerical prediction criteria, the share of numerical-prediction points earned, mostly fold-error credit min(p, t)/max(p, t) (a twofold error earns half) and otherwise the task's stated tolerance or threshold, averaged over ten responses per task and then equally over those tasks. It is normalized within the component rather than being an additive share of the overall score, so it is a diagnostic of medchembench. Categorical predictions are not part of it. OpenAI's MedChemBench (November 20, 2026) poses 99 medicinal-chemistry problems built from 117 recent papers and patents (median publication year 2025), each giving the biological context, compound structures and experimental evidence (structure drawings, assay tables, plots, protein–ligand views) up to a decision point in a drug program; a model answers each in one turn, without tools.
No signal or difficulty estimate. Held out of the fit by design.
| # | Model | Setting | Score | Source |
|---|---|---|---|---|
| 1 | GPT-6 Astra | max | 47.6 | |
| 2 | GPT-6.1 Sol | max | 46.8 | |
| 3 | GPT-5.5 | high | 46.0 | |
| 4 | GPT-6.1 Terra | max | 45.5 | |
| 5 | GPT-6 Sol | max | 45.1 | |
| 6 | GPT-5.6 Sol | max | 44.5 | |
| 7 | GPT-5.6 Terra | max | 41.8 | |
| 8 | GPT-5.4 | x-high | 40.8 | |
| 9 | GPT-5.2 | x-high | 40.0 | |
| 10 | GPT-6 Luna | max | 40.0 | |
| 11 | GPT-5.6 Luna | max | 38.6 |
Benchmarks outside its family whose model rankings track it most reliably (Spearman ρ over at least 10 shared models).