Benchmarks
326 sources · 1,256 models × 2,956 benchmarks · 154,484 scoresThe Harness Leaderboard's run of Terminal-Bench 2.1 (commit d49e28f1) on 87 of its 89 tasks, comparing coding-agent harnesses on fixed open models in Harbor; qemu-startup and qemu-alpine-ssh are left out because they need KVM, which Hugging Face's sandboxes lack. Each task scores the mean of three attempts and the rate is the mean over tasks. Infrastructure failures are retried and excluded, and trials that looked up a task's answer key count as failures. Kept as a component beside the 89-task average task accuracy.
241st strongest of 779 fitted benchmarks
Signal-to-noise +8.74 for the Terminal-Bench 2.1 family
Another benchmark represents this family in the fit.
| # | Model | Setting | Score | Source |
|---|---|---|---|---|
| 1 | DeepSeek V4.1 Flash | low · Harness Leaderboard; mini-swe-agent 2.4.6 in Harbor | 76.2 | |
| 2 | GLM-5.3 Flash | high · Harness Leaderboard; Pi 1.2.0 in Harbor; assumed a 128,000-token window and 16,384-token replies | 69.3 | |
| 3 | Flint | always on · Harness Leaderboard; Kimi Code 2.2.0 in Harbor; told every model holds 1,000,000 tokens | 32.2 |