SRE Bench — Capability Score
Code · 2026-09-21
Mean fraction of six verified subtasks completed per SRE Bench instance, across 262 instances (1,572 subtasks). Incomplete instances contribute their actual completed subtasks. The evaluation unit is the instance because its six subtasks are dependent.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-6 Astra | 63.9 |
| GPT-5.6 Sol | 59.5 |
| Gemini 4 Argon | 57.8 |
| GPT-6 Sol | 57.3 |
| GPT-6.1 Sol | 56.3 |
| GPT-5.6 Terra | 26.8 |
| Gemini 3.8 Flash | 19.7 |
| GLM-5.3 | 19.5 |
| Claude Haiku 5.5 | 18.1 |
| Muse Wonder Full | 17.7 |
| Grok 4.7 | 17.6 |
| GPT-5.5 | 17.0 |
| MiMo-V2.6-Flash | 15.9 |
| MiMo-V2.6-Pro | 15.8 |
| DeepSeek V4.1 Flash | 14.2 |
| Gemini 3.7 Flash | 14.2 |
| Muse Spark 1.3 | 12.3 |
| GPT-6 Luna | 11.1 |
| Hy4 Preview | 10.8 |
| DeepSeek V4 Pro 0813 | 10.6 |
| Sonnet 5 | 7.7 |
| Step 5 Preview | 6.8 |
| GLM-5.3 Flash | 6.5 |
| GPT-5.6 Luna | 6.4 |
| Grok 4.5 | 6.3 |
| GLM-5.2 | 3.4 |
| Muse Spark 1.2 | 1.5 |
| Gemini 3.1 Pro Preview | 1.0 |
| Inkling | 0.3 |