← All benchmarks

STEMBench120 v1.0.0 (closed-book)

Science

Ulam's STEMBench120 v1.0.0 run closed-book through model APIs with no model-side tools: 120 private, scoped research tasks with defined full targets, 20 each in biology, chemistry, physics, astronomy, Earth science and engineering. Each response gets an authoritative integer progress level from 0 (no usable progress) to 4 (full target), and an experimental quality index Q refines levels 1-3 into a fractional score s + 0.95Q. The score is the complete-run mean fractional score over all 120 tasks, with unusable or missing responses scored 0; up to three retries are allowed and the grader is not named.

Top models (higher is better)