GSM8K
Math · 2021-10-27
A grade-school math word problem benchmark focused on multi-step arithmetic reasoning and exact-match solutions.
Top models (higher is better)
| Model | Score |
|---|---|
| Ternary Bonsai 2 27B (GGUF) | 96.7 |
| Claude 3.5 Sonnet (June 2024) | 96.4 |
| Opus 3 | 95.0 |
| Claude 3 Sonnet | 92.3 |
| Gemini 1.5 Pro (May 2024) | 90.8 |
| Claude 3 Haiku | 88.9 |
| Gemini 1.0 Ultra | 88.9 |
| Gemini 1.5 Flash (May 2024 report) | 86.2 |
| PaLM 2-L | 80.0 |
| Gemini 1.0 Pro | 77.9 |
| Grok 1 (November 2023) | 62.9 |