Social IQa (SIQA)
General QA · 2019-04-22
Social IQa tests commonsense reasoning about people's actions and their social and emotional consequences with three-option multiple-choice questions about everyday situations. Results are reported on the 1,954-question development split, because test labels are withheld, as answer accuracy. Higher is better.
Top models (higher is better)
| Model | Score |
|---|---|
| Gemma 3 27B PT | 54.9 |
| Llama 3.1 405B | 53.7 |
| Gemma 3 12B PT | 53.4 |
| LLaMA 65B | 52.3 |
| RecurrentGemma 9B | 52.3 |
| Llama 3.1 70B | 52.2 |
| Gemma 3 4B PT | 51.9 |
| Gemma 7B | 51.8 |
| RecurrentGemma 2B | 51.8 |
| Mixtral 8x22B | 51.6 |
| Chinchilla | 51.3 |
| Llama 2 34B | 50.9 |
| Llama 2 70B | 50.7 |
| Gopher (280B) | 50.6 |
| LLaMA-13B | 50.4 |
| LLaMA 33B | 50.4 |
| Llama 2 13B | 50.3 |
| Falcon 40B | 50.1 |
| Gemma 3n E4B | 50.0 |
| Gemma 2B | 49.7 |
| Llama 3.1 8B | 49.5 |
| Gemma 3 1B PT | 48.9 |
| LLaMA 7B | 48.9 |
| MPT-30B | 48.9 |
| Gemma 3n E2B | 48.8 |
| MPT-7B | 48.5 |
| Llama 2 7B | 48.3 |
| Falcon 7B | 47.2 |
| Mistral 7B v0.1 Base | 47.0 |