BFCL v3 Overall Accuracy

Name: BFCL v3 Overall Accuracy
Published: 2024-09-19

Agents · 2024-09-19

Overall accuracy on version 3 of the Berkeley Function-Calling Leaderboard, which added multi-turn and multi-step function-calling categories to the v2 single-turn non-live and live sets (about 4,400 entries in all). The overall score aggregates the non-live, live, multi-turn and relevance/irrelevance categories. It is version-specific and is not pooled with BFCL v4, which adds web-search and memory agentic categories.

Top models (higher is better)

Model	Score
Sonnet 4	73.3
GPT-4o (2024-11-20)	72.5
o3	72.4
Qwen3 30B A3B Thinking 2507	72.4
Qwen3-Next-80B-A3B-Thinking	72.0
Llama 3.3 Nemotron Super 49B V1.5	71.8
Qwen3 VL 32B Thinking	71.7
Qwen3 4B Thinking 2507	71.2
Qwen3.5-4B	71.0
Qwen3 235B A22B Instruct 2507	70.9
Qwen3-235B-A22B	70.8
Qwen3 32B	70.3
Qwen3 Next 80B A3B Instruct	70.3
Qwen3 VL 32B Instruct	70.2
Qwen3-30B-A3B	69.1
Qwen3-Coder-480B-A35B-Instruct	68.7
Gemini 2.5 Flash	68.6
Qwen3 VL 30B A3B Thinking	68.6
Nemotron 3.5 Bolt E4B BF16	68.5
o1 (2024-12-17)	67.8
Qwen3-VL-235B-A22B-Instruct	67.7
Nemotron Cascade 14B Thinking	67.5
Qwen3 VL 4B Thinking	67.3
O4 Mini	67.2
Gemini 2.5 Pro	67.2
Nemotron Cascade 8B Thinking	67.0
Nemotron Nano 12B V2	67.0
Nemotron Nano 9B V2	66.9
ChatGPT-4o Latest observed 2025-03-27	66.5
QwQ-32B	66.4
Gemma 4 E4B IT	66.4
Qwen3 VL 30B A3B Instruct	66.3
Qwen3 4B	65.9
Kimi K2 Instruct	65.2
Qwen3 30B A3B Instruct 2507	65.1
DeepSeek-V3-0324	64.7
o3-mini	64.6
Nemotron Cascade 8B	64.4
Qwen3 Omni 30B A3B Instruct	64.4
GPT-4o Mini	64.0