Benchmarks
347 sources · 1,258 models × 3,038 benchmarks · 158,912 scoresAutoResearchExam's hidden-test AUARC over only the first 6 hours of each 24-hour research window, from the same single run per task: the time-weighted average of the hidden-test reward held over hours 0-6, averaged over the 29 tasks. Bespoke Labs' November 10 analysis of how models explore ideas reports it as early AUARC; it rewards reaching good solutions quickly, where the 24-hour headline also credits late gains.
154th strongest of 798 fitted benchmarks
Signal-to-noise +11.36 for the AutoResearchExam (hidden-test AUARC) family
Another benchmark represents this family in the fit.
| # | Model | Setting | Score | Source |
|---|---|---|---|---|
| 1 | GPT-6 Astra | max · Terminus 2 | 0.5 | |
| 2 | Gemini 4 Argon | high · Terminus 2 | 0.5 | |
| 3 | Claude Opus 5.5 | max · Terminus 2 | 0.5 | |
| 4 | Claude Fable 5.1 | max · Terminus 2 | 0.5 | |
| 5 | GPT-5.6 Sol | max · Terminus 2 | 0.5 | |
| 6 | Grok 4.7 | x-high · Terminus 2 | 0.4 | |
| 7 | Claude Opus 5 | max · Terminus 2 | 0.4 | |
| 8 | Grok 4.6 | x-high · Terminus 2 | 0.4 | |
| 9 | Gemini 3.8 Flash | high · Terminus 2 | 0.4 | |
| 10 | Kimi K3 | max · Terminus 2 | 0.3 | |
| 11 | Muse Spark 1.3 | x-high · Terminus 2 | 0.2 |
Benchmarks outside its family whose model rankings track it most reliably (Spearman ρ over at least 10 shared models).