Atlas

Benchmarks

AutoResearchExam (hidden-test AUARC, first 6 hours)

AutoResearchExam's hidden-test AUARC over only the first 6 hours of each 24-hour research window, from the same single run per task: the time-weighted average of the hidden-test reward held over hours 0-6, averaged over the 29 tasks. Bespoke Labs' November 10 analysis of how models explore ideas reports it as early AUARC; it rewards reaching good solutions quickly, where the 24-hour headline also credits late gains.

Models
11
Top score
0.5
Leader
GPT-6 Astra
Updated
Nov 10, 2026
Resolution
—
Saturation
—
Signal
81stpercentile
195 benchmarks from 0.8 to 3.5224 benchmarks from 3.5 to 6.1133 benchmarks from 6.1 to 8.898 benchmarks from 8.8 to 11.566 benchmarks from 11.5 to 14.126 benchmarks from 14.1 to 16.815 benchmarks from 16.8 to 19.59 benchmarks from 19.5 to 22.18 benchmarks from 22.1 to 24.84 benchmarks from 24.8 to 27.55 benchmarks from 27.5 to 30.12 benchmarks from 30.1 to 32.84 benchmarks from 32.8 to 35.45 benchmarks from 35.4 to 38.11 benchmark from 38.1 to 40.82 benchmarks from 43.4 to 46.11 benchmark from 62.1 to 64.8

154th strongest of 798

Signal-to-noise +11.36 for the AutoResearchExam (hidden-test AUARC) family

Difficulty
Not estimated

Another benchmark represents this family in the fit.

Organizations
11 dated models · 3 frontier modelsClosed weightsOpen weights
Data: Bespoke Labsatlas.village.dev

AutoResearchExam (hidden-test AUARC, first 6 hours) leaderboard

#ModelSettingScoreSource
1GPT-6 Astramax · Terminus 2
0.5
2Gemini 4 Argonhigh · Terminus 2
0.5
3Claude Opus 5.5max · Terminus 2
0.5
4Claude Fable 5.1max · Terminus 2
0.5
5GPT-5.6 Solmax · Terminus 2
0.5
6Grok 4.7x-high · Terminus 2
0.4
7Claude Opus 5max · Terminus 2
0.4
8Grok 4.6x-high · Terminus 2
0.4
9Gemini 3.8 Flashhigh · Terminus 2
0.4
10Kimi K3max · Terminus 2
0.3
11Muse Spark 1.3x-high · Terminus 2
0.2

Related benchmarks

Loading Atlas data…