← All benchmarks

Toloka Arena (v3)

Agents

Toloka's agentic intelligence index (toloka.ai/arena) on its current task set: private, τ-bench-style multi-turn customer-service tasks in which the agent works with MCP-like tools, live databases and written business rules while a simulated customer converses with it, across the 11 industry tool-use domains the page lists. The score is the composite pass^5, the probability that all five independent trials of a task succeed. Models run at reasoning medium, 16,384 max tokens and temperature 0.6 unless a label states otherwise. Toloka does not publish the task count (its cost data imply about 100 tasks) or how the composite weights the domains. The page has loaded this refreshed task set, which it says has refined criteria, since at least October 12, 2026; it is not comparable with the earlier toloka-arena-v1 cohort.

Top models (higher is better)