Terminal-Bench 1.0
Code · 2025-05-19
Terminal-Bench 1.0 (the original terminal-bench-core task set, about 80 tasks) evaluates AI agents completing real tasks in sandboxed Linux terminal environments, from compiling code and configuring systems to training models, with each task checked by its own test script. Scores report the percentage of tasks solved. The agent harness (for example Terminus, Terminus 2 or a vendor's own framework) is a run-level setting recorded on the config. It is distinct from the rebuilt 89-task Terminal-Bench 2.0. Higher is better.
Top models (higher is better)
| Model | Score |
|---|---|
| Sonnet 4.5 | 50.0 |
| Kimi K2 Thinking | 47.1 |
| Opus 4.1 | 46.5 |
| Kimi K2 Instruct 0905 | 44.5 |
| GPT-5 | 43.8 |
| Opus 4 | 43.2 |
| Haiku 4.5 | 41.0 |
| DeepSeek-V3.2-Exp | 37.7 |
| Kimi K2 Instruct | 37.5 |
| DeepSeek-V3.1-Terminus | 36.7 |
| Sonnet 4 | 36.4 |
| Claude 3.7 Sonnet | 35.2 |
| DeepSeek-V3.1 | 31.3 |
| DeepSeek-V3-0324 | 13.3 |
| DeepSeek-R1-0528 | 5.7 |