PIQA
General QA · 2019-11-26
A physical commonsense benchmark where models choose the more feasible solution to everyday problems.
Top models (higher is better)
| Model | Score |
|---|---|
| Nemotron 3 Super 120B A12B Base | 85.5 |
| Kimi K2 Base | 85.5 |
| Llama 3 70B | 85.0 |
| Nemotron-H 56B Base 8K | 85.0 |
| PaLM 2-L | 85.0 |
| DeepSeek-V3-Base | 84.7 |
| Nemotron 3 Nano 30B A3B Base | 84.3 |
| Nemotron-H 47B Base 8K | 83.9 |
| Nemotron 3 Ultra 550B A55B Base | 83.8 |
| DeepSeek LLM 67B Base | 83.6 |
| Mixtral 8x7B | 83.6 |
| Qwen1.5 72B | 83.3 |
| Gemma 3 27B PT | 83.3 |
| Stable Beluga 2 | 83.3 |
| PaLM 2-M | 83.2 |
| LLaMA 65B | 82.8 |
| DeepSeek LLM 67B Chat | 82.6 |
| Nemotron Nano 12B V2 Base | 82.5 |
| Falcon 40B | 82.4 |
| Nemotron-4 15B | 82.4 |
| Nemotron-H 8B Base 8K | 82.2 |
| PaLM 2-S | 82.2 |
| MPT-30B | 81.9 |
| Llama 2 34B | 81.9 |
| Nemotron Nano 9B V2 Base | 81.8 |
| Chinchilla | 81.8 |
| Gopher (280B) | 81.8 |
| Gemma 3 12B PT | 81.8 |
| RecurrentGemma 9B | 81.3 |
| Gemma 3n E4B | 81.0 |
| Llama 3.1 8B | 81.0 |
| MPT-7B | 80.6 |
| InternLM 20B | 80.3 |
| DeepSeekMoE 16B Base | 80.2 |
| LLaMA-13B | 80.1 |
| Qwen 14B | 79.9 |
| LLaMA 7B | 79.8 |
| DeepSeekMoE 16B Chat | 79.7 |
| Gemma 3 4B PT | 79.6 |
| Nemotron-H 4B Base 8K | 79.4 |