Atlas

Benchmarks

SWE-Marathon v1.2

Current SWE-Marathon v1.2 cohort of 20 scored long-horizon software engineering tasks on rebuilt task environments (every published sample task's environment digest differs from v1.1's). 15 tasks are published as samples and 5 are held out. Abundant runs up to eight trials per task, unevenly spread, and scores each configuration as the mean of its per-task binary pass rates, so every task counts the same. Kept separate from the archived v1.0 and v1.1 cohorts.

Models
13
Top score
86.8
Leader
Claude Fable 5.2
Updated
Nov 20, 2026
Resolution
~2 mo
Sharp
Saturation
~May 2027
Projected
Signal
70thpercentile
141 benchmarks from 0.8 to 2.9196 benchmarks from 2.9 to 5.0131 benchmarks from 5.0 to 7.1105 benchmarks from 7.1 to 9.282 benchmarks from 9.2 to 11.365 benchmarks from 11.3 to 13.327 benchmarks from 13.3 to 15.421 benchmarks from 15.4 to 17.58 benchmarks from 17.5 to 19.68 benchmarks from 19.6 to 21.76 benchmarks from 21.7 to 23.84 benchmarks from 23.8 to 25.95 benchmarks from 25.9 to 28.03 benchmarks from 28.0 to 30.12 benchmarks from 30.1 to 32.14 benchmarks from 32.1 to 34.21 benchmark from 34.2 to 36.33 benchmarks from 36.3 to 38.41 benchmark from 38.4 to 40.52 benchmarks from 40.5 to 42.61 benchmark from 42.6 to 44.71 benchmark from 44.7 to 46.81 benchmark from 48.9 to 50.9

248th strongest of 818

Signal-to-noise +9.09

Difficulty
69thpercentile
1 benchmark from 5.8 to 14.71 benchmark from 23.5 to 32.42 benchmarks from 32.4 to 41.21 benchmark from 41.2 to 50.12 benchmarks from 50.1 to 58.99 benchmarks from 58.9 to 67.88 benchmarks from 67.8 to 76.718 benchmarks from 76.7 to 85.523 benchmarks from 85.5 to 94.441 benchmarks from 94.4 to 103.232 benchmarks from 103.2 to 112.141 benchmarks from 112.1 to 121.049 benchmarks from 121.0 to 129.875 benchmarks from 129.8 to 138.760 benchmarks from 138.7 to 147.5122 benchmarks from 147.5 to 156.4135 benchmarks from 156.4 to 165.3124 benchmarks from 165.3 to 174.139 benchmarks from 174.1 to 183.021 benchmarks from 183.0 to 191.88 benchmarks from 191.8 to 200.72 benchmarks from 200.7 to 209.54 benchmarks from 209.5 to 218.4

255th hardest of 818

Capability needed ≲ 161.0

Organizations
13 dated models · 3 frontier modelsClosed weightsOpen weights
Data: SWE-Marathon Maintainersatlas.village.dev

SWE-Marathon v1.2 leaderboard

Related benchmarks

Loading Atlas data…