Atlas

Benchmarks

SPU (Science Prediction Universe)

UniPat's SPU tests whether an agent can predict the results of real experiments before running them. Each task comes from one wet-lab study published in a Nature Portfolio journal between January 2025 and August 2026: the agent ranks the candidate experiments in each comparison group by a hidden measured outcome, such as yield, conductivity or binding, with eight hours, one GPU and web access through audited tools that end the run if it finds the source study. An offline grader scores each group by Spearman correlation with the measurements and each task as a fixed-weight sum of its groups, so a task scores from −1 to 1 and a random valid ranking averages 0. The 100 public tasks (85 papers; chemistry, materials science, biology and medicine) were each run once per configuration in Claude Code or Codex. The score is the grader's default mean over all 100 official runs, in which a refusal, a missing file or a malformed file scores −1, so every configuration shares the same denominator.

Models
10
Top score
0.4
Leader
GPT-6 Astra
Updated
Nov 11, 2026
Resolution
~1.1 years
Blunt
Saturation
No trend
No estimate
Signal
41stpercentile
195 benchmarks from 0.8 to 3.5224 benchmarks from 3.5 to 6.1133 benchmarks from 6.1 to 8.898 benchmarks from 8.8 to 11.566 benchmarks from 11.5 to 14.126 benchmarks from 14.1 to 16.815 benchmarks from 16.8 to 19.59 benchmarks from 19.5 to 22.18 benchmarks from 22.1 to 24.84 benchmarks from 24.8 to 27.55 benchmarks from 27.5 to 30.12 benchmarks from 30.1 to 32.84 benchmarks from 32.8 to 35.45 benchmarks from 35.4 to 38.11 benchmark from 38.1 to 40.82 benchmarks from 43.4 to 46.11 benchmark from 62.1 to 64.8

475th strongest of 798

Signal-to-noise +4.84

Difficulty
98thpercentile
1 benchmark from 5.5 to 14.41 benchmark from 14.4 to 23.22 benchmarks from 32.1 to 41.01 benchmark from 41.0 to 49.82 benchmarks from 49.8 to 58.79 benchmarks from 58.7 to 67.67 benchmarks from 67.6 to 76.418 benchmarks from 76.4 to 85.324 benchmarks from 85.3 to 94.241 benchmarks from 94.2 to 103.031 benchmarks from 103.0 to 111.939 benchmarks from 111.9 to 120.850 benchmarks from 120.8 to 129.676 benchmarks from 129.6 to 138.560 benchmarks from 138.5 to 147.4118 benchmarks from 147.4 to 156.2124 benchmarks from 156.2 to 165.1120 benchmarks from 165.1 to 174.039 benchmarks from 174.0 to 182.820 benchmarks from 182.8 to 191.710 benchmarks from 191.7 to 200.61 benchmark from 200.6 to 209.44 benchmarks from 209.4 to 218.3

17th hardest of 798

Capability needed ≳ 189.7

Organizations
10 dated models · 2 frontier modelsClosed weightsOpen weights
Data: UniPat AIatlas.village.dev

SPU (Science Prediction Universe) leaderboard

Related benchmarks

Most correlated

Benchmarks outside its family whose model rankings track it most reliably (Spearman ρ over at least 10 shared models).

Loading Atlas data…