OmnilingualGAIA2
Agents · 2026-08-09
OmnilingualGAIA2 extends four GAIA2 agent capabilities (search, execution, ambiguity and adaptability) to ten languages: Spanish, French, Portuguese, Indonesian, Italian, Turkish, Chinese, German, Hindi and Japanese. Tasks and the simulated universe are both in the target language, and a verifier with a multilingual LLM judge grades actions and responses. The headline score macro-averages pass@1 over the four capabilities and then over the ten languages.
Top models (higher is better)
| Model | Score |
|---|---|
| Muse Glimmer 30B | 60.7 |
| Qwen3.6 27B | 45.0 |
| Gemma 4 31B IT | 37.5 |