← All benchmarks

τ^τ-bench release v1 (telecom)

Agents · 2026-09-04

The telecom block of τ^τ-bench release v1, Sierra's agent-construction benchmark in which a coding harness driving a builder model must build a customer-service agent from a business's records, a simulated client and a client API, and is scored by deploying that agent against held-out τ³-bench-style simulated users (a task scores the agent's mean reward less any budget-overage penalty, floored at zero). Its 6 tasks are telecom technical support (line diagnostics and repair flows). The score is the mean over the 6 tasks from one construction trial each; with the other three domain blocks it makes up the 53-task overall score (hyper-tau-bench-v1). Like that score, it rates the model as the builder of the agent, not as the customer-service agent that τ-bench, τ²-bench and τ³-bench score.

Benchmark source: arXiv

Top models (higher is better)