Atlas

Benchmarks

SWE Atlas v1.1 - Overall

Scale

SWE Atlas v1.1's index, the simple mean of its Codebase QnA, Test Writing and Refactoring scores. 284 tasks from 15 production repositories in 6 languages, run in Docker with standard shell tools and re-run for v1.1 with a GPT-6.1 Sol judge (majority of 3 votes), refined grading criteria and restricted network access. Scale says v1.1 scores aren't directly comparable to v1.0, so the v1.0 boards stay separate.

Models
15
Top score
77.4
Leader
Claude Opus 5.5
Updated
—
Resolution
~1 mo
Sharp
Saturation
~Jun 2027
Projected
Signal
92ndpercentile
205 benchmarks from 0.8 to 3.6219 benchmarks from 3.6 to 6.4134 benchmarks from 6.4 to 9.393 benchmarks from 9.3 to 12.155 benchmarks from 12.1 to 14.925 benchmarks from 14.9 to 17.713 benchmarks from 17.7 to 20.55 benchmarks from 20.5 to 23.37 benchmarks from 23.3 to 26.27 benchmarks from 26.2 to 29.01 benchmark from 29.0 to 31.83 benchmarks from 31.8 to 34.66 benchmarks from 34.6 to 37.43 benchmarks from 37.4 to 40.21 benchmark from 40.2 to 43.11 benchmark from 45.9 to 48.71 benchmark from 65.6 to 68.4

65th strongest of 779

Signal-to-noise +15.70

Difficulty
63rdpercentile
1 benchmark from 5.9 to 14.81 benchmark from 23.6 to 32.52 benchmarks from 32.5 to 41.31 benchmark from 41.3 to 50.22 benchmarks from 50.2 to 59.08 benchmarks from 59.0 to 67.97 benchmarks from 67.9 to 76.819 benchmarks from 76.8 to 85.623 benchmarks from 85.6 to 94.541 benchmarks from 94.5 to 103.331 benchmarks from 103.3 to 112.239 benchmarks from 112.2 to 121.148 benchmarks from 121.1 to 129.973 benchmarks from 129.9 to 138.859 benchmarks from 138.8 to 147.6118 benchmarks from 147.6 to 156.5127 benchmarks from 156.5 to 165.3113 benchmarks from 165.3 to 174.235 benchmarks from 174.2 to 183.117 benchmarks from 183.1 to 191.99 benchmarks from 191.9 to 200.81 benchmark from 200.8 to 209.64 benchmarks from 209.6 to 218.5

286th hardest of 779

Capability needed 157.8

15 dated models · 3 frontier modelsClosed weightsOpen weights
Data: Scale AIatlas.village.dev
#ModelSettingScoreSource
1Claude Opus 5.5not reported
77.4
Scale Labs SWE Atlas - Overall leaderboard (snapshot 2026-11-09)
2Claude Fable 5.1not reported
74.1
Scale Labs SWE Atlas - Overall leaderboard (snapshot 2026-11-09)
3Gemini 4 Argonnot reported
70.1
Scale Labs SWE Atlas - Overall leaderboard (snapshot 2026-11-09)
4GPT-6 Astranot reported
69.4
Scale Labs SWE Atlas - Overall leaderboard (snapshot 2026-11-09)
5Claude Sonnet 5.5not reported
67.5
Scale Labs SWE Atlas - Overall leaderboard (snapshot 2026-11-09)
6GPT-6.1 Solnot reported
65.9
Scale Labs SWE Atlas - Overall leaderboard (snapshot 2026-11-09)
7Muse Wondernot reported
64.8
Scale Labs SWE Atlas - Overall leaderboard (snapshot 2026-11-09)
8Grok 4.7not reported
62.1
Scale Labs SWE Atlas - Overall leaderboard (snapshot 2026-11-09)
9Inkling 2 Smallnot reported
61.9
Scale Labs SWE Atlas - Overall leaderboard (snapshot 2026-11-09)
10Kimi K3not reported
61.0
Scale Labs SWE Atlas - Overall leaderboard (snapshot 2026-11-09)
11Claude Haiku 5.5not reported
59.4
Scale Labs SWE Atlas - Overall leaderboard (snapshot 2026-11-09)
12GPT-6.1 Terranot reported
57.7
Scale Labs SWE Atlas - Overall leaderboard (snapshot 2026-11-09)
13Gemini 3.8 Flashnot reported
57.3
Scale Labs SWE Atlas - Overall leaderboard (snapshot 2026-11-09)
14GPT-6 Lunanot reported
50.8
Scale Labs SWE Atlas - Overall leaderboard (snapshot 2026-11-09)
15Muse Spark Opennot reported
42.3
Scale Labs SWE Atlas - Overall leaderboard (snapshot 2026-11-09)
Loading Atlas data…