Benchmarks
326 sources · 1,256 models × 2,956 benchmarks · 154,484 scoresThe repair level of AutoRLBench's second task list, which 24 of its 32 failed reinforcement-learning runs support. The agent repairs a writable copy of the run's workspace and submits a report without launching training; the harness reruns the experiment on the repaired workspace and grades only that fresh run against a metric threshold, and repairs that game the reward or rely on fetched upstream material fail. Each model runs each task once.
No signal or difficulty estimate. Held out of the fit by design.
| # | Model | Setting | Score | Source |
|---|---|---|---|---|
| 1 | GPT-6.1 Sol | max · Codex | 57.9 | |
| 2 | GLM-5.3 | max · Claude Code | 52.6 | |
| 3 | GPT-6 Astra | max · Codex | 52.6 | |
| 4 | Kimi K3.1 Pro | max · Claude Code | 52.6 | |
| 5 | Claude Fable 5.1 | max · Claude Code | 42.1 | |
| 6 | Kimi K3 | max · Claude Code | 10.5 |