WinoGrande
General QA · 2019-07-24
WinoGrande is a large-scale adversarial Winograd-style commonsense reasoning benchmark with 44k pronoun-resolution problems.
Top models (higher is better)
| Model | Score |
|---|---|
| Nemotron-4 340B Base | 89.5 |
| Opus 3 | 88.5 |
| GPT-4 (source unspecified) | 87.5 |
| C4AI Command R+ | 85.4 |
| Qwen2-72B | 85.1 |
| Llama 3.1 70B Instruct | 85.1 |
| Llama 3.1 Nemotron 51B Instruct | 84.5 |
| Nemotron-H 47B Base 8K | 83.9 |
| Qwen1.5 110B | 83.5 |
| PaLM 2-L | 83.0 |
| Qwen2.5 32B | 82.0 |
| DeepSeek-V4-Pro-Base | 81.5 |
| Qwen1.5 32B | 81.5 |
| Kimi Linear 48B A3B Base | 81.5 |
| Command R | 81.5 |
| Qwen2.5 14B | 81.0 |
| Qwen2.5-Coder-32B | 80.8 |
| DeepSeek-V4-Flash-Base | 79.5 |
| Qwen2-57B-A14B | 79.5 |
| Nemotron 3 Ultra 550B A55B Base | 79.3 |
| Nemotron Nano 12B V2 Base | 79.2 |
| PaLM 2-M | 79.2 |
| Nemotron 3 Super 120B A12B Base | 78.9 |
| Gemma 3 27B PT | 78.8 |
| Nemotron-4 15B | 78.0 |
| PaLM 2-S | 77.9 |
| Llama 4 Maverick | 77.6 |
| LLaMA 65B | 77.0 |
| Qwen2-7B | 77.0 |
| Falcon 40B | 76.9 |
| Qwen2.5-Coder-14B | 76.8 |
| Llama 2 34B | 76.7 |
| DeepSeek LLM 67B Chat | 76.0 |
| Qwen2.5 7B | 75.9 |
| Nemotron Nano 9B V2 Base | 75.3 |
| Claude 3 Sonnet | 75.1 |
| Chinchilla | 74.9 |
| Moonlight 16B A3B | 74.6 |
| Phi-2 | 74.4 |
| Gemma 3 12B PT | 74.3 |