GSM8K (Maj1@8)
Math · 2021-10-27
GSM8K accuracy with self-consistency: the majority answer over 8 sampled chain-of-thought solutions (maj@8), usually with 8-shot prompting. Kept separate from single-sample GSM8K.
Top models (higher is better)
| Model | Score |
|---|---|
| Mixtral 8x22B Instruct | 90.8 |
| Mixtral 8x22B | 88.4 |
| Mixtral 8x7B | 74.4 |
| Llama 2 70B | 69.6 |
| LLaMA 33B | 44.1 |
| Llama 2 13B | 34.3 |
| Llama 2 7B | 16.0 |