Toloka Arena (v3) pass^1
Agents
Per-trial success rate on Toloka Arena's current (v3) task set, from the same five trials per task as the composite pass^5 headline: pass^1 is the share of single trials that succeed, the mean over the five trials.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Opus 5.5 | 84.4 |
| Claude Fable 5.1 | 82.6 |
| Gemini 4 Argon | 80.5 |
| Claude Sonnet 5.5 | 79.8 |
| GPT-6.1 Sol | 79.4 |
| GPT-6 Astra | 78.4 |
| Muse Wonder | 78.4 |
| GPT-6 Sol | 77.4 |
| GPT-5.6 Sol | 76.6 |
| Grok 4.6 | 76.4 |
| Claude Opus 5 | 76.0 |
| Gemini 3.8 Flash | 75.6 |
| Gemini 3.7 Flash | 72.6 |
| GPT-6.1 Terra | 72.0 |
| Beam 1.0 501B A23B | 64.2 |
| GPT-6 Luna | 62.6 |
| Qwen3.8 27B | 58.9 |
| Grok 4.7 | 57.4 |
| MiniMax M3.1 Flash | 51.1 |
| Claude Haiku 5.5 | 49.6 |
| Sonnet 5 | 45.0 |
| Kimi K3 | 41.0 |
| Muse Spark Open | 39.6 |
| Nemotron 3.5 Super VL 120B A12B (precision unspecified) | 25.8 |
| Gemini 3.1 Pro Preview | 22.4 |