TruthfulQA (Unspecified Metric)
General QA · 2021-09-08
TruthfulQA results for which the reporting source does not state whether the MC1, MC2 or generation metric was used. TruthfulQA has 817 questions across 38 categories designed to elicit imitative falsehoods. Values are retained as reported but kept apart from metric-specific comparisons.
Top models (higher is better)
| Model | Score |
|---|---|
| Qwen2.5 72B | 60.4 |
| Qwen1.5 72B | 59.6 |
| Qwen2.5 14B | 58.4 |
| Qwen2.5 32B | 57.8 |
| Qwen2-57B-A14B | 57.7 |
| Qwen1.5 32B | 57.4 |
| Qwen2.5 7B | 56.4 |
| Qwen2.5-Coder-14B | 55.2 |
| Qwen2-72B | 54.8 |
| Qwen2.5-Coder-32B | 54.2 |
| Qwen2-7B | 54.2 |
| Mixtral 8x7B | 51.1 |
| Qwen1.5 7B | 51.1 |
| Mixtral 8x22B | 51.0 |
| Qwen2.5 Coder 7B | 50.6 |
| Qwen1.5 110B | 49.6 |
| Qwen2.5-Coder-3B | 49.2 |
| Qwen2.5 3B | 48.9 |
| Qwen2.5 1.5B | 46.6 |
| Qwen2-1.5B | 45.9 |
| Llama 3 70B | 45.6 |
| Gemma 2 9B | 45.3 |
| Gemma 7B | 44.8 |
| Phi-2 | 44.5 |
| Qwen2.5 Coder 1.5B | 44.0 |
| Llama 3 8B | 44.0 |
| Qwen2.5 Coder 0.5B | 42.7 |
| DeepSeek-Coder-Base 1.3B | 42.7 |
| CodeQwen1.5 7B | 42.2 |
| DeepSeek-V2 | 42.2 |
| StarCoder2-7B | 42.0 |
| StarCoder2-3B | 40.5 |
| DeepSeek-Coder-Base 6.7B | 40.2 |
| Qwen2.5 0.5B | 40.2 |
| Gemma 2 27B | 40.1 |
| DeepSeek-Coder-Base 33B | 40.0 |
| Qwen2-0.5B | 39.7 |
| Qwen1.5 1.8B | 39.4 |
| DeepSeek-Coder-V2-Lite-Base | 38.8 |
| StarCoder2-15B | 37.9 |