Atlas
Benchmarks
326 sources · 1,256 models × 2,956 benchmarks · 154,484 scoresSWE Atlas v1.1 - Codebase QnA
Codebase QnA component of SWE Atlas v1.1, on questions that need runtime analysis and multi-file reasoning. Part of a suite of 284 tasks from 15 production repositories in 6 languages, run in Docker with standard shell tools and re-run for v1.1 with a GPT-6.1 Sol judge (majority of 3 votes), refined grading criteria and restricted network access. Scale says v1.1 scores aren't directly comparable to v1.0, so the v1.0 boards stay separate.
Models
15
Top score
77.4
Leader
Claude Opus 5.5
Updated
—
Resolution
—
Saturation
—
Signal
92ndpercentile
65th strongest of 779 fitted benchmarks
Signal-to-noise +15.70 for the SWE Atlas v1.1 - Overall family
Difficulty
Not estimated
Another benchmark represents this family in the fit.
| # | Model | Setting | Score | Source |
|---|---|---|---|---|
| 1 | Claude Opus 5.5 | not reported | 77.4 | |
| 2 | Inkling 2 Small | not reported | 74.2 | |
| 3 | Claude Fable 5.1 | not reported | 73.4 | |
| 4 | Grok 4.7 | not reported | 73.4 | |
| 5 | Muse Wonder | not reported | 73.4 | |
| 6 | Claude Sonnet 5.5 | not reported | 71.8 | |
| 7 | GPT-6.1 Sol | not reported | 71.0 | |
| 8 | GPT-6 Astra | not reported | 70.2 | |
| 9 | Gemini 4 Argon | not reported | 65.3 | |
| 10 | Kimi K3 | not reported | 65.3 | |
| 11 | Claude Haiku 5.5 | not reported | 62.1 | |
| 12 | GPT-6.1 Terra | not reported | 54.0 | |
| 13 | GPT-6 Luna | not reported | 50.8 | |
| 14 | Muse Spark Open | not reported | 50.0 | |
| 15 | Gemini 3.8 Flash | not reported | 45.2 |
