← All benchmarks

τ^τ-bench release v1 (airline_plus)

Agents · 2026-09-04

The airline_plus block of τ^τ-bench release v1, Sierra's agent-construction benchmark in which a coding harness driving a builder model must build a customer-service agent from a business's records, a simulated client and a client API, and is scored by deploying that agent against held-out τ³-bench-style simulated users (a task scores the agent's mean reward less any budget-overage penalty, floored at zero). Its 6 tasks serve Meridian Airlines (booking, changes, cancellations and compensation), a rebuilt variant of the τ³-bench airline domain with a new brand, values and policies, so that memorized public policies do not transfer. The score is the mean over the 6 tasks from one construction trial each; with the other three domain blocks it makes up the 53-task overall score (hyper-tau-bench-v1). Like that score, it rates the model as the builder of the agent, not as the customer-service agent that τ-bench, τ²-bench and τ³-bench score.

Benchmark source: arXiv

Top models (higher is better)