Atlas

Benchmarks

Agent Initiative Benchmark

Deepnote's Agent Initiative Benchmark tests whether an AI teammate knows when to help without being asked. The agent sits in the Slack of PageMantle, a synthetic company with six years of records across 11 connected services (CRM, product analytics, billing, a warehouse, an issue tracker, docs and Slack), and faces 50 workplace situations across eight task types: it reads the conversation, may investigate with search and read tools, including SQL, and then posts, replies, sends a direct message or stays silent, within 180 seconds of model response time. Some situations call for silence. Two blinded LLM evaluators score each run from 0 to 10, and the score is their mean averaged over the 50 situations; a run that times out when help was needed is a miss. Every model runs at high reasoning effort with the same instructions and tools.

Models
10
Top score
9.0
Leader
GPT-6 Astra
Updated
—
Resolution
~7 mo
Coarse
Saturation
~Apr 2027
Projected
Signal
24thpercentile
195 benchmarks from 0.8 to 3.5224 benchmarks from 3.5 to 6.1133 benchmarks from 6.1 to 8.898 benchmarks from 8.8 to 11.566 benchmarks from 11.5 to 14.126 benchmarks from 14.1 to 16.815 benchmarks from 16.8 to 19.59 benchmarks from 19.5 to 22.18 benchmarks from 22.1 to 24.84 benchmarks from 24.8 to 27.55 benchmarks from 27.5 to 30.12 benchmarks from 30.1 to 32.84 benchmarks from 32.8 to 35.45 benchmarks from 35.4 to 38.11 benchmark from 38.1 to 40.82 benchmarks from 43.4 to 46.11 benchmark from 62.1 to 64.8

606th strongest of 798

Signal-to-noise +3.43

Difficulty
45thpercentile
1 benchmark from 5.5 to 14.41 benchmark from 14.4 to 23.22 benchmarks from 32.1 to 41.01 benchmark from 41.0 to 49.82 benchmarks from 49.8 to 58.79 benchmarks from 58.7 to 67.67 benchmarks from 67.6 to 76.418 benchmarks from 76.4 to 85.324 benchmarks from 85.3 to 94.241 benchmarks from 94.2 to 103.031 benchmarks from 103.0 to 111.939 benchmarks from 111.9 to 120.850 benchmarks from 120.8 to 129.676 benchmarks from 129.6 to 138.560 benchmarks from 138.5 to 147.4118 benchmarks from 147.4 to 156.2124 benchmarks from 156.2 to 165.1120 benchmarks from 165.1 to 174.039 benchmarks from 174.0 to 182.820 benchmarks from 182.8 to 191.710 benchmarks from 191.7 to 200.61 benchmark from 200.6 to 209.44 benchmarks from 209.4 to 218.3

442nd hardest of 798

Capability needed ≲ 146.5

Organizations
10 dated models · 2 frontier modelsClosed weightsOpen weights
Data: Deepnoteatlas.village.dev

Agent Initiative Benchmark leaderboard

#ModelSettingScoreSource
1GPT-6 Astrahigh · Deepnote Agent Initiative Benchmark harness; Slack plus search and read tools across 11 simulated PageMantle services; 180-second model-time budget
9.0
2GPT-6.1 Solhigh · Deepnote Agent Initiative Benchmark harness; Slack plus search and read tools across 11 simulated PageMantle services; 180-second model-time budget
8.7
3Claude Fable 5.2high · Deepnote Agent Initiative Benchmark harness; Slack plus search and read tools across 11 simulated PageMantle services; 180-second model-time budget
8.2
4Claude Opus 5.5high · Deepnote Agent Initiative Benchmark harness; Slack plus search and read tools across 11 simulated PageMantle services; 180-second model-time budget
7.8
5Claude Fable 5.1high · Deepnote Agent Initiative Benchmark harness; Slack plus search and read tools across 11 simulated PageMantle services; 180-second model-time budget
7.5
6Claude Sonnet 5.5high · Deepnote Agent Initiative Benchmark harness; Slack plus search and read tools across 11 simulated PageMantle services; 180-second model-time budget
6.7
7GPT-6 Lunahigh · Deepnote Agent Initiative Benchmark harness; Slack plus search and read tools across 11 simulated PageMantle services; 180-second model-time budget
6.4
8Claude Haiku 5.5high · Deepnote Agent Initiative Benchmark harness; Slack plus search and read tools across 11 simulated PageMantle services; 180-second model-time budget
6.0
9Gemini 4 Argonhigh · Deepnote Agent Initiative Benchmark harness; Slack plus search and read tools across 11 simulated PageMantle services; 180-second model-time budget
5.6
10Grok 4.7high · Deepnote Agent Initiative Benchmark harness; Slack plus search and read tools across 11 simulated PageMantle services; 180-second model-time budget
4.5

Related benchmarks

Most correlated

Benchmarks outside its family whose model rankings track it most reliably (Spearman ρ over at least 10 shared models).

Loading Atlas data…