INCLUDE
General QA · 2024-11-29
INCLUDE evaluates multilingual knowledge and reasoning with four-option multiple-choice questions collected from local academic and professional exams in 44 languages, so questions carry regional rather than translated content. Scores report accuracy averaged across languages; release tables do not identify the full or lite subset.
Top models (higher is better)
| Model | Score |
|---|---|
| Gemini 3 Pro Preview | 90.5 |
| GPT-5.2 | 87.5 |
| Opus 4.6 | 87.4 |
| Opus 4.5 | 86.2 |
| Qwen3.7-Max (source unspecified) | 86.2 |
| DeepSeek-V4-Pro | 86.1 |
| Qwen3.5 397B A17B | 85.6 |
| Qwen3.6 Plus (2026-04-02) | 85.1 |
| GLM-5 | 84.9 |
| GLM-5.1 | 84.3 |
| Kimi K2.6 | 84.2 |
| Kimi K2.5 | 83.3 |
| Qwen3.7-Plus (source unspecified) | 83.0 |
| Qwen3.5 122B A10B | 82.8 |
| Qwen3 Max Thinking (2026-01-23) | 82.3 |
| GPT-5 Mini | 81.8 |
| Qwen3.5-27B | 81.6 |
| Qwen3-235B-A22B | 81.0 |
| Qwen3.5 35B A3B | 79.7 |
| Qwen3-Next-80B-A3B-Thinking | 78.3 |
| Qwen3.5-9B | 75.6 |
| Qwen3 30B A3B Thinking 2507 | 74.4 |
| gpt-oss-120b | 74.0 |
| Qwen3.5-4B | 71.0 |
| gpt-oss-20b | 65.3 |
| Qwen3 4B Thinking 2507 | 64.4 |
| Qwen3.5-2B | 55.4 |
| Qwen3 1.7B | 51.8 |
| Qwen3.5-0.8B | 40.6 |