MMLU Pro - Math
Math · 2024-06-03
The Math split of MMLU Pro. Its guessing floor averages the reciprocal option count across all 1,351 subject questions in official dataset revision b189ec765aa7ed75c8acfea42df31fdae71f97be. This component remains distinct from the parent aggregate.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Fable 5.1 | 97.3 |
| Gemini 3.5 Flash | 96.3 |
| Gemini 3.8 Flash | 96.2 |
| Gemini 3.1 Pro Preview | 96.0 |
| Gemini 3.6 Flash | 96.0 |
| Gemini 3.7 Flash | 96.0 |
| Gemini 3 Pro Preview | 96.0 |
| Muse Spark 1.1 | 96.0 |
| Muse Spark 1.2 | 96.0 |
| Qwen3.7-Max (source unspecified) | 96.0 |
| Opus 4.6 | 95.9 |
| Sonnet 5 | 95.9 |
| Grok 4.5 | 95.9 |
| Grok 4.6 | 95.9 |
| Opus 4.7 | 95.9 |
| Opus 4.8 | 95.9 |
| GLM-5.2 | 95.9 |
| Gemini 3 Flash Preview | 95.7 |
| Kimi K2.5 | 95.7 |
| Kimi K2.6 | 95.6 |
| Kimi K3 | 95.6 |
| Nemotron 3 Ultra 550B A55B (NVFP4) | 95.6 |
| Qwen3.5 Plus (2026-02-15) | 95.6 |
| Qwen3 Max Thinking (2026-01-23) | 95.6 |
| GLM-5.1 | 95.5 |
| Qwen3.6 Plus (2026-04-02) | 95.5 |
| Qwen3.8-Max | 95.5 |
| DeepSeek-V4-Pro | 95.3 |
| GPT-5.6 Sol | 95.3 |
| Inkling-Small | 95.3 |
| GLM-5.3 Flash | 95.2 |
| GLM-5.3 | 95.2 |
| DeepSeek V4 Pro 0813 | 95.1 |
| Grok 4.20 | 95.1 |
| DeepSeek-V3.2 | 95.0 |
| GPT-5.2 | 95.0 |
| Muse Spark | 95.0 |
| Qwen3 Max (rolling alias) | 95.0 |
| Sonnet 4.5 | 95.0 |
| Inkling | 94.9 |