Atlas

Benchmarks

SWE-Together

SWE-Together (Meta) rebuilds 109 repository-level coding tasks in 27 repositories from real multi-turn sessions between users and coding agents. The agent starts from the session's first user message, and a user simulator (Gemini 3.1 Pro on this board) replays the original user's follow-ups, clarifications and corrections while the agent works in a sandbox. An agentic Claude Opus 4.6 judge scores each final patch against a frozen, weighted task rubric, and a run solves its task at a judge score of 0.85 or above. The score is pass@1, the share of tasks solved averaged over two runs per task, with every model in the opencode harness. Scores are comparable only within one user simulator.

Models
18
Top score
69.3
Leader
Claude Fable 5.1
Updated
—
Resolution
~5 mo
Useful
Saturation
2029–2031+
Projected
Signal
27thpercentile
195 benchmarks from 0.8 to 3.5224 benchmarks from 3.5 to 6.1133 benchmarks from 6.1 to 8.898 benchmarks from 8.8 to 11.566 benchmarks from 11.5 to 14.126 benchmarks from 14.1 to 16.815 benchmarks from 16.8 to 19.59 benchmarks from 19.5 to 22.18 benchmarks from 22.1 to 24.84 benchmarks from 24.8 to 27.55 benchmarks from 27.5 to 30.12 benchmarks from 30.1 to 32.84 benchmarks from 32.8 to 35.45 benchmarks from 35.4 to 38.11 benchmark from 38.1 to 40.82 benchmarks from 43.4 to 46.11 benchmark from 62.1 to 64.8

584th strongest of 798

Signal-to-noise +3.69

Difficulty
33rdpercentile
1 benchmark from 5.5 to 14.41 benchmark from 14.4 to 23.22 benchmarks from 32.1 to 41.01 benchmark from 41.0 to 49.82 benchmarks from 49.8 to 58.79 benchmarks from 58.7 to 67.67 benchmarks from 67.6 to 76.418 benchmarks from 76.4 to 85.324 benchmarks from 85.3 to 94.241 benchmarks from 94.2 to 103.031 benchmarks from 103.0 to 111.939 benchmarks from 111.9 to 120.850 benchmarks from 120.8 to 129.676 benchmarks from 129.6 to 138.560 benchmarks from 138.5 to 147.4118 benchmarks from 147.4 to 156.2124 benchmarks from 156.2 to 165.1120 benchmarks from 165.1 to 174.039 benchmarks from 174.0 to 182.820 benchmarks from 182.8 to 191.710 benchmarks from 191.7 to 200.61 benchmark from 200.6 to 209.44 benchmarks from 209.4 to 218.3

534th hardest of 798

Capability needed ≲ 134.6

Organizations
18 dated models · 3 frontier modelsClosed weightsOpen weights

SWE-Together leaderboard

Related benchmarks

Loading Atlas data…