NanoGPT Speedrun Frontier (human record gap closed)
Agents · 2026-08-15
Prime Intellect's NanoGPT Speedrun Frontier runs a model unattended in an agent harness for up to about nine days on the modded-nanogpt track 3 optimizer speedrun. On one 8xH200 node whose only network route is the model's API, the agent changes the training recipe of a 124M-parameter GPT to reach validation loss 3.28 in as few training steps as possible, and a frozen verifier accepts a record only if the mean loss of eight trainings on fixed seeds beats 3.27859. The score is the share of the gap between the 3,290-step tuned baseline and the 2,600-step human record that the model's best validated record closes, (3290 - record) / 690, in percent, so it would exceed 100 only by beating the human record. The publisher launches at least three seeds for most models and reports each model's best validated run across seeds and harnesses, so a score is a best of several runs on one task. It is a different measurement from OpenAI's internal NanoGPT self-improvement figure (nanogpt-openai-reported), which uses track 1 on a single H100 for under a day.
Benchmark source: Primeintellect