Toolathlon-Verified
Agents · 2026-06-30
Toolathlon-Verified evaluates agents on the verified 108-task Toolathlon suite across real-world applications and tools. The official trajectory release reports Pass@1 as mean task success over three runs per model.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Opus 5 | 80.6 |
| Claude Fable 5 | 79.3 |
| MiMo-V2.6-Pro | 76.9 |
| Kimi K3 | 76.5 |
| GPT-5.6 Sol | 74.9 |
| Sonnet 5 | 74.7 |
| DeepSeek V4 Pro 0813 | 74.1 |
| MiMo-V2.6-Flash | 73.6 |
| Qwen3.8-Flash-Next | 73.5 |
| GPT-5.5 | 73.5 |
| GLM-5.3 | 73.0 |
| Muse Spark Open | 72.8 |
| Nanbeige4.3-33B | 70.4 |
| GPT-5.6 Luna | 67.9 |
| Gemini 3.5 Flash | 67.3 |
| Gemini 3.1 Pro Preview | 61.1 |
| GLM-5.2 | 59.9 |
| Kimi K2.7 Code | 58.0 |
| Kimi K2.6 | 58.0 |
| Gemini 3.5 Flash-Lite | 57.1 |
| Inkling-Small | 54.4 |
| DeepSeek-V4-Flash | 50.9 |
| Laguna S 2.1 | 49.7 |
| MiMo-V2.5 | 49.1 |
| MiniMax M2.7 | 47.5 |
| Inkling | 45.5 |
| Qwen3.5 397B A17B | 40.7 |
| Nemotron 3 Ultra 550B A55B | 34.3 |
| Kimi K2.5 | 33.0 |
| Haiku 4.5 | 26.9 |