Atlas
Benchmarks
326 sources · 1,256 models × 2,956 benchmarks · 154,484 scoresSWE Atlas v1.1 - Test Writing
Test Writing component of SWE Atlas v1.1, on writing production-grade tests for repository behavior. Part of a suite of 284 tasks from 15 production repositories in 6 languages, run in Docker with standard shell tools and re-run for v1.1 with a GPT-6.1 Sol judge (majority of 3 votes), refined grading criteria and restricted network access. Scale says v1.1 scores aren't directly comparable to v1.0, so the v1.0 boards stay separate.
Models
15
Top score
78.9
Leader
Claude Fable 5.1
Updated
—
Resolution
—
Saturation
—
Signal
92ndpercentile
65th strongest of 779 fitted benchmarks
Signal-to-noise +15.70 for the SWE Atlas v1.1 - Overall family
Difficulty
Not estimated
Another benchmark represents this family in the fit.
| # | Model | Setting | Score | Source |
|---|---|---|---|---|
| 1 | Claude Fable 5.1 | not reported | 78.9 | |
| 2 | Claude Opus 5.5 | not reported | 77.8 | |
| 3 | Gemini 4 Argon | not reported | 77.8 | |
| 4 | Gemini 3.8 Flash | not reported | 71.1 | |
| 5 | Claude Sonnet 5.5 | not reported | 67.8 | |
| 6 | GPT-6.1 Sol | not reported | 66.7 | |
| 7 | GPT-6 Astra | not reported | 66.7 | |
| 8 | Grok 4.7 | not reported | 64.4 | |
| 9 | Claude Haiku 5.5 | not reported | 63.3 | |
| 10 | GPT-6.1 Terra | not reported | 63.3 | |
| 11 | Kimi K3 | not reported | 63.3 | |
| 12 | Muse Wonder | not reported | 61.1 | |
| 13 | Inkling 2 Small | not reported | 58.1 | |
| 14 | GPT-6 Luna | not reported | 54.4 | |
| 15 | Muse Spark Open | not reported | 41.1 |
