TriviaQA
General QA · 2017-05-09
An open-domain question answering benchmark with challenging trivia questions paired with evidence documents.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude 2 | 87.5 |
| Claude 1.3 | 86.7 |
| DeepSeek-V2-Chat | 86.7 |
| PaLM 2-L | 86.1 |
| DeepSeek-V4-Pro-Base | 85.6 |
| Gemma 3 27B PT | 85.5 |
| Kimi K2 Base | 85.1 |
| DeepSeek-V4-Flash-Base | 82.8 |
| Llama 3.1 405B | 82.7 |
| DeepSeek-Coder-V2-Instruct | 82.3 |
| PaLM 2-M | 81.7 |
| Llama 3 70B | 81.6 |
| DeepSeek LLM 67B Chat | 81.5 |
| PaLM 540B | 81.4 |
| Mistral Small 3 24B Base 2501 | 80.3 |
| Mixtral 8x22B Instruct | 80.0 |
| Qwen1.5 72B Chat | 79.6 |
| Llama 4 Maverick | 79.3 |
| Claude Instant 1.1 | 78.9 |
| DeepSeek LLM 67B Base | 78.9 |
| Kimi Linear 48B A3B Base | 75.2 |
| PaLM 2-S | 75.2 |
| Ministral 3 14B Base 2512 | 74.9 |
| Mistral NeMo Base 2407 | 73.8 |
| Qwen1.5 72B | 73.1 |
| PaLM 62B | 72.7 |
| Chinchilla | 72.3 |
| RecurrentGemma 9B | 70.5 |
| Gemma 3n E4B | 70.2 |
| Llama 3 70B Instruct | 69.1 |
| LLaMA 33B | 68.5 |
| Ministral 3 8B Base 2512 | 68.1 |
| Gemma 3 4B PT | 65.8 |
| DeepSeek-V2-Lite-Chat | 65.2 |
| DeepSeekMoE 16B Base | 64.8 |
| DeepSeek-V2-Lite | 64.2 |
| Gemma 7B | 63.4 |
| DeepSeekMoE 16B Chat | 63.3 |
| Gopher (280B) | 61.3 |
| Llama 3 8B | 61.0 |