MCP Atlas public (Muse Glimmer report, Gemini 2.5 Pro judge)
Agents · 2026-10-17
Meta's internal MCP-Atlas runs in the Muse Glimmer report cover the 500 public tasks over 20+ MCP tool servers. Gemini 2.5 Pro judges whether the final answer fulfils the ground-truth claims, a task passes above a 0.75 claim score, and results average four runs. The report states no turn or tool-call budget or retry handling, so this source-specific protocol stays separate from Scale's original and April 2026 leaderboards.
Top models (higher is better)
| Model | Score |
|---|---|
| Muse Glimmer 30B | 77.4 |
| Qwen3.6 27B | 62.5 |
| Gemma 4 31B IT | 54.2 |