Benchmarks
347 sources · 1,258 models × 3,038 benchmarks · 158,912 scoresInfrathon (University of Washington and AI2) gives coding agents 25 GPU infrastructure-optimization tasks, 11 operator, 5 module and 9 system tasks such as kernels, attention modules and serving stacks, with 12- or 24-hour budgets on H100s, and measures each submission's best throughput or latency in the task's own unit. Because the task metrics have no common baseline, the headline ranks the six evaluated agents on each task and averages a rank score of 100 × (6 − rank) / 5. Scores are relative to the paper's six-agent cohort, so they are diagnostic and not comparable with other cohorts.
No signal or difficulty estimate. Held out of the fit by design.
| # | Model | Setting | Score | Source |
|---|---|---|---|---|
| 1 | Claude Opus 5.5 | max · Claude Code | 91.2 | |
| 2 | GPT-6 Astra | max · Codex | 75.2 | |
| 3 | GPT-6.1 Sol | max · Codex | 54.4 | |
| 4 | Qwen3.8 Flash | max · Codex | 40.8 | |
| 5 | GLM-5.3 Flash | max · Codex | 8.0 |