ARC-Easy
General QA · 2018-03-14
Easy science multiple-choice benchmark from AI2 Reasoning Challenge.
Top models (higher is better)
| Model | Score |
|---|---|
| DeepSeek-V3-Base | 98.9 |
| Qwen2.5 72B | 98.4 |
| Llama 3.1 405B | 98.4 |
| DeepSeek-V2-Chat | 98.1 |
| Llama 3 70B | 97.9 |
| DeepSeek-V2 | 97.6 |
| DeepSeek-Coder-V2-Instruct | 97.4 |
| Mixtral 8x22B | 97.3 |
| Qwen1.5 72B | 97.1 |
| Mixtral 8x22B Instruct | 97.1 |
| Llama 3 70B Instruct | 96.9 |
| Qwen1.5 72B Chat | 96.8 |
| Stable Beluga 2 | 91.2 |
| Qwen 14B | 90.3 |
| PaLM 2-L | 89.7 |
| Gemma 3 27B PT | 89.0 |
| DeepSeek-Coder-V2-Lite-Instruct | 88.9 |
| Gemma 2 27B | 88.6 |
| Gemma 3 12B PT | 88.3 |
| PaLM 2-M | 88.0 |
| InternLM 20B | 86.1 |
| PaLM 2-S | 85.6 |
| Qwen 7B | 84.0 |
| Nemotron-H 8B Base 8K | 83.6 |
| Mixtral 8x7B | 83.1 |
| Gemma 3 4B PT | 82.4 |
| Gemma 3n E4B | 81.6 |
| Nemotron-H 4B Base 8K | 81.6 |
| Gemma 7B | 81.5 |
| Nemotron-4 15B | 80.9 |
| Llama 2 34B | 79.4 |
| Falcon 40B | 79.2 |
| RecurrentGemma 9B | 78.8 |
| InternLM 7B | 78.7 |
| MPT-30B | 76.5 |
| Gemma 3n E2B | 75.8 |
| Gemma 2B | 73.2 |
| Gemma 3 1B PT | 73.0 |
| RecurrentGemma 2B | 72.9 |
| DeepSeek-V2-Lite | 70.9 |