MMVU
Multimodal · 2025-01-21
MMVU measures expert-level, multi-discipline video understanding with multiple-choice and open-ended questions written by domain experts across 27 subjects in science, healthcare, humanities and engineering. Release tables report accuracy and do not always identify the validation or test split.
Top models (higher is better)
| Model | Score |
|---|---|
| Qwen3.8-Max | 82.4 |
| Kimi K3 | 82.1 |
| GPT-5.5 | 81.7 |
| GPT-5.6 Sol | 81.2 |
| GPT-5.2 | 80.8 |
| Kimi K2.5 | 80.4 |
| Opus 4.8 | 79.2 |
| Gemini 3.1 Pro Preview | 77.9 |
| GPT-5.6 Luna | 77.6 |
| Gemini 3 Pro Preview | 77.5 |
| Opus 4.5 | 77.3 |
| Qwen3.7-Plus (source unspecified) | 76.6 |
| Gemini 3.5 Flash-Lite | 76.3 |
| Qwen3.5 397B A17B | 75.4 |
| Qwen3.5 122B A10B | 74.7 |
| Qwen3.5-27B | 73.3 |
| Qwen3.5 35B A3B | 72.3 |
| Qwen3 VL 235B A22B Thinking | 71.1 |
| Sonnet 4.5 | 70.6 |
| GPT-5 Mini | 69.8 |
| Qwen3.5-9B | 67.8 |
| Qwen3.5-4B | 67.5 |
| Qwen3.5-Omni-Plus | 67.5 |
| Qwen3.5 Plus (2026-02-15) | 67.1 |
| Gemma 4 12B IT | 65.5 |
| Gemini 2.5 Flash-Lite | 65.3 |
| GPT-5 Nano | 63.1 |
| Qwen2.5-VL-72B-Instruct | 62.9 |
| Qwen3.5-Omni-Flash | 62.7 |
| GPT-4o Mini | 61.6 |
| Gemma 3 27B IT | 61.3 |
| Kimi-VL-A3B-Thinking-2506 | 57.5 |
| Gemma 3 12B IT | 57.0 |
| Kimi-VL-A3B-Thinking | 53.0 |
| LFM3-VL-2B | 52.9 |
| Qwen2.5-VL-7B-Instruct | 50.1 |
| Gemma 4 E2B IT | 47.7 |
| Llama 3.2 11B Vision Instruct | 44.4 |
| Qwen3.5-0.8B | 34.3 |