Benchmarks
357 sources · 1,263 models × 3,079 benchmarks · 162,127 scoresAbundant AI's FrontierRSI asks whether an AI agent can do the research and engineering needed to improve AI itself. Its 27 open-ended tasks, five in pre-training, eight in agentic post-training and harness design, five in inference and serving and nine in H100 GPU kernels, each state a goal to optimize (held-out loss, latency, throughput or a held-out success rate) with no target to reach, and give the agent 45 minutes to 8 hours on one or two H100 or A100 GPUs. A trial's reward is relative to the best valid trial any evaluated model reached on the task: the best scores 1, a result 10% or more worse scores 0, with a linear ramp between, and a trial that fails a correctness, accuracy or resource check scores 0. A model's task score is the mean of two trials (avg@2) and its score the mean over the 27 tasks, in percent. Because the reference is the cohort's own best trial, a stronger new model lowers everyone else's score, so this is a diagnostic. Unrelated to Synthetic Sciences' FrontierML, whose released tasks were briefly named FrontierRSI.
No signal or difficulty estimate. Held out of the fit by design.
| # | Model | Setting | Score | Source |
|---|---|---|---|---|
| 1 | Claude Opus 5.5 | max · Claude Code | 34.3 | |
| 2 | Claude Fable 5.2 | max · Claude Code | 33.0 | |
| 3 | Gemini 4 Carbon | high · Antigravity CLI | 29.8 | |
| 4 | Gemini 4 Argon | high · Antigravity CLI | 23.6 | |
| 5 | GPT-6.1 Sol | max · Codex | 23.5 | |
| 6 | GPT-6 Astra | max · Codex | 20.9 | |
| 7 | Kimi K3.1 Pro | max · Claude Code | 13.6 | |
| 8 | Muse Wonder | max · Muse Code | 12.0 | |
| 9 | Grok 4.7 | x-high · Grok Build | 8.9 |