ComplexFuncBench
Agents · 2025-01-17
ComplexFuncBench (Zhipu AI and Tsinghua University) tests multi-step and constrained function calling in long-context travel-booking scenarios: 1,000 samples in which a model must chain real API calls, fill parameters under user constraints and summarise the results. The paper scores overall success rate alongside call-level accuracy, completeness and correctness; release posts report a single accuracy without always naming which, so this is diagnostic. Higher is better.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-4o (2024-11-20) | 66.5 |
| GPT-4.1 | 65.5 |
| GPT-4.5 | 63.0 |
| GPT-4.1 Mini | 49.3 |
| o1 (2024-12-17) | 47.6 |
| GPT-4o Mini | 38.6 |
| o3-mini | 17.6 |
| GPT-4.1 Nano | 5.7 |