Global MMLU Lite
General QA · 2024-12-04
Global MMLU Lite is a Cohere Labs multilingual and multicultural knowledge benchmark. This primary headline uses Artificial Analysis' 16-language aggregate of culturally sensitive and culturally agnostic questions. Google's verified 15-language evaluation and release-card results with unverified language rosters are separate diagnostic identities; their aggregate cohorts are not assumed equivalent.
Top models (higher is better)
| Model | Score |
|---|---|
| Gemini 3.1 Pro Preview | 93.2 |
| Gemini 3 Flash Preview | 92.8 |
| Gemini 3 Pro Preview | 92.2 |
| Opus 4.6 | 92.2 |
| Opus 4.5 | 91.3 |
| GPT-5 | 90.7 |
| GPT-5.1 | 90.6 |
| Sonnet 4.6 | 90.5 |
| Gemini 2.5 Pro | 90.3 |
| Qwen3.5 397B A17B | 90.0 |
| GPT-5.2 | 89.8 |
| GPT-5.1-Codex | 89.7 |
| Grok 4 | 89.5 |
| Grok 4.20 | 89.5 |
| GPT-5.4 Mini | 89.4 |
| Sonnet 4.5 | 89.3 |
| Qwen3.5 122B A10B | 89.1 |
| Gemini 2.5 Flash Preview (09-2025) | 88.8 |
| DeepSeek-V3.2-Speciale | 88.4 |
| Nova 2 Pro (Preview) | 87.6 |
| GPT-5 Mini | 87.4 |
| Qwen3.5-27B | 86.8 |
| DeepSeek-V3.2-Exp | 86.7 |
| DeepSeek-V3.2 | 86.5 |
| ERNIE 5.0 Thinking Preview | 86.5 |
| Qwen3.5 35B A3B | 86.4 |
| DeepSeek-R1-0528 | 86.0 |
| Grok 4 Fast | 85.9 |
| Grok 4.1 Fast | 85.6 |
| GLM 4.6 | 85.6 |
| Qwen3 Max Preview | 85.5 |
| Qwen3 235B A22B Thinking 2507 | 85.3 |
| GPT-5.1-Codex mini | 85.0 |
| MiMo-V2-Flash | 84.4 |
| MiniMax M2.5 | 84.2 |
| Kimi K2.5 | 84.0 |
| MiniMax M2.1 | 84.0 |
| Gemini 2.5 Flash-Lite Preview (09-2025) | 83.8 |
| Haiku 4.5 | 83.4 |
| Qwen3-Max (2025-09-23) | 83.3 |