Benchmarks
331 sources · 1,258 models × 2,980 benchmarks · 155,643 scoresThe share of Longtail-SWE's 202 issue-resolution tasks solved in all three attempts, a measure of consistency, in the authors' SWE-agent-like scaffold.
No signal or difficulty estimate. Too few models have been measured on this benchmark to estimate its difficulty.
| # | Model | Setting | Score | Source |
|---|---|---|---|---|
| 1 | Claude Opus 5.5 | high · SWE-rebench swelike scaffold; 1M context | 73.8 | |
| 2 | GPT-6.1 Sol | high · SWE-rebench swelike scaffold; 1M context | 73.3 | |
| 3 | GPT-5.6 Sol | high · SWE-rebench swelike scaffold; 1M context | 63.4 | |
| 4 | GPT-5.6 Luna | x-high · SWE-rebench swelike scaffold; 1M context | 56.9 | |
| 5 | Kimi K3 | max · SWE-rebench swelike scaffold; 262K context | 48.5 | |
| 6 | GLM-5.3 Flash | not reported · SWE-rebench swelike scaffold; 262K context | 23.8 | |
| 7 | Qwen3.6 27B | thinking · SWE-rebench swelike scaffold; 262K context | 9.4 |