τ³ Banking Knowledge - BM25 retrieval
Agents
The 97-task banking_knowledge domain of τ³-bench run with the benchmark's default BM25 knowledge-base retrieval and the official GPT-5.2 (low reasoning effort) user simulator. Agents retrieve bank policy from a large knowledge base and carry out multi-step support workflows with tools; the score is the mean binary task reward (pass^1) over repeated trials. Kept as a protocol component beside the all-tools runs of Artificial Analysis, Atria and Fireworks; release sources may not state the trial count.
Top models (higher is better)
| Model | Score |
|---|---|
| LFM2.5-24B-A2B | 8.0 |
| Qwen3.6 35B A3B | 7.6 |
| Nemotron 3.5 Lightning 30B A3B (precision unspecified) | 7.0 |
| Gemma 4 26B A4B IT | 4.5 |