Benchmarks
347 sources · 1,258 models × 3,038 benchmarks · 158,912 scoresInferenceBench gives a coding agent a base model, a GPU environment and a two-hour budget to build an OpenAI-compatible inference server, and scores the final submitted server's speedup over a vanilla PyTorch baseline on four scenarios: prefill latency (time to first token), decode latency (time per output token), throughput under burst, Poisson and constant-rate traffic, and a balanced all-in-one. A server that fails the correctness checks or the integrity audit for reward hacking, is unreachable, or regresses scores at the baseline (1.0). Each scenario is run three times, and the score is the geometric mean of the four scenario speedups. This is the v1.0.1 prompt, the leaderboard's default view.
No signal or difficulty estimate. This benchmark has no declared score range, so it cannot be placed on the 0-1 scale the capability curve requires.
| # | Model | Setting | Score | Source |
|---|---|---|---|---|
| 1 | Claude Opus 5.5 | max · Claude Code | 12.1 | |
| 2 | Claude Sonnet 5.5 | max · Claude Code | 12.1 | |
| 3 | Claude Fable 5.2 | max · Claude Code | 10.3 | |
| 4 | Claude Fable 5.1 | not reported · Claude Code | 9.8 | |
| 5 | Claude Opus 5 | not reported · Claude Code | 8.9 | |
| 6 | Claude Fable 5 | low · Claude Code | 8.7 | |
| 7 | GPT-6 Astra | not reported · Codex | 7.9 | |
| 8 | GPT-6 Sol | not reported · Codex | 7.6 | |
| 9 | GPT-5.6 Sol | ultra · Codex | 7.3 | |
| 10 | GLM-5.2 | max · Claude Code | 7.0 | |
| 11 | Grok 4.6 | not reported · Grok Build | 6.7 | |
| 12 | Sonnet 5 | not reported · Claude Code | 6.4 | |
| 13 | Kimi K2.7 Code | always on · OpenCode | 6.3 | |
| 14 | Kimi K3 | not reported · OpenCode | 5.7 | |
| 15 | GLM-5.3 | max · Claude Code | 5.0 | |
| 16 | Muse Spark 1.3 | not reported · OpenCode | 4.5 | |
| 17 | Claude Haiku 5.5 | max · Claude Code | 4.4 | |
| 18 | Grok 4.5 | not reported · Grok Build | 3.7 |
Benchmarks outside its family whose model rankings track it most reliably (Spearman ρ over at least 10 shared models).