MMMU Pro
Multimodal · 2024-09-04
MMMU-Pro is a more robust version of MMMU that removes text-only-answerable items, expands the candidate options, and adds a vision-only input setting. Scores are reported as answer accuracy.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Opus 5.5 | 86.6 |
| GPT-6 Astra | 86.4 |
| Claude Fable 5.1 | 86.1 |
| Claude Sonnet 5.5 | 85.8 |
| GPT-6.1 Sol | 85.1 |
| Gemini 3.7 Flash | 84.9 |
| GPT-6.1 Terra | 84.7 |
| GPT-5.6 Sol | 84.6 |
| Gemini 3.8 Flash | 84.5 |
| Gemini 3.1 Pro Preview | 84.3 |
| Kimi K3.1 Pro | 84.1 |
| Claude Opus 5 | 84.0 |
| Gemini 3.5 Flash | 83.9 |
| GPT-5.5 | 83.2 |
| Qwen3.8 Max (0902) | 82.8 |
| GPT-6 Sol | 82.5 |
| Qwen3.8-Max | 82.3 |
| GPT-5.4 | 82.1 |
| GPT-5.6 Terra | 82.0 |
| Muse Spark 1.2 | 81.4 |
| Muse Spark 1.1 | 81.3 |
| Muse Spark Open | 80.9 |
| DeepSeek V4.1 Flash 1017 | 80.6 |
| Muse Spark | 80.5 |
| Grok 4.6 | 80.5 |
| Qwen3.7-Plus (source unspecified) | 80.5 |
| GPT-5.2 | 80.4 |
| Gemini 3 Pro Preview | 80.2 |
| Gemini 3 Flash Preview | 79.9 |
| Qwen3.8-Flash-Next | 79.8 |
| GPT-6 Luna | 79.7 |
| GPT-5.6 Luna | 79.5 |
| Kimi K2.6 | 79.4 |
| Apodex 1.1 | 79.2 |
| MiniMax M3.1 Flash | 79.1 |
| Claude Haiku 5.5 | 79.1 |
| Gemini 3.5 Flash-Lite | 79.0 |
| Ling 3.0 Flash VL | 79.0 |
| MiniMax M3 | 78.9 |
| Opus 4.7 | 78.8 |