Benchmarks
357 sources · 1,263 models × 3,079 benchmarks · 162,127 scoresCurrent SWE-Marathon v1.2 cohort of 20 scored long-horizon software engineering tasks on rebuilt task environments (every published sample task's environment digest differs from v1.1's). 15 tasks are published as samples and 5 are held out. Abundant runs up to eight trials per task, unevenly spread, and scores each configuration as the mean of its per-task binary pass rates, so every task counts the same. Kept separate from the archived v1.0 and v1.1 cohorts.
248th strongest of 818 fitted benchmarks
Signal-to-noise +9.09
255th hardest of 818 fitted benchmarks
Capability needed ≲ 161.0
| # | Model | Setting | Score | Source |
|---|---|---|---|---|
| 1 | Claude Fable 5.2 | max · Claude Code | 86.8 | |
| 2 | Claude Opus 5.5 | max · Claude Code | 81.3 | |
| 3 | Claude Sonnet 5.5 | max · Claude Code | 80.6 | |
| 4 | GPT-6.2 Astra | max · Codex | 77.5 | |
| 5 | GPT-6 Astra | max · Codex | 68.8 | |
| 6 | GPT-6.2 Sol | max · Codex | 66.9 | |
| 7 | Gemini 4 Carbon | high · Antigravity CLI | 66.7 | |
| 8 | Gemini 4 Neon | x-high · Antigravity CLI | 62.5 | |
| 9 | GPT-6.1 Sol | max · Codex | 60.4 | |
| 10 | Gemini 4 Hydrogen | x-high · Antigravity CLI | 58.1 | |
| 11 | Gemini 4 Argon | high · Antigravity CLI | 54.5 | |
| 12 | Claude Haiku 5.5 | max · Claude Code | 48.1 | |
| 13 | Muse Wonder | max · Muse Code | 38.1 |
Benchmarks outside its family whose model rankings track it most reliably (Spearman ρ over at least 10 shared models).