SWE Atlas - Codebase QnA (pass@3)
Code · 2026-03-04
This aggregation of the SWE Atlas Codebase QnA track reports pass@3 on repository-level questions requiring runtime analysis and multi-file reasoning. It is kept separate from single-run resolve-rate observations on the parent track.
Top models (higher is better)
| Model | Score |
|---|---|
| Macaron-V1-Venti | 49.5 |
| MiniMax M3 | 37.9 |
| Qwen3.7-Max (source unspecified) | 22.6 |