TruthfulQA generation - % informative
Safety · 2021-09-08
TruthfulQA's generation task asks 817 questions written so that repeating common misconceptions gives false answers. A fine-tuned GPT-info model rates each free-form answer, and this score is the percentage judged informative, the companion to the truthfulness rate.
Top models (higher is better)
| Model | Score |
|---|---|
| Llama 2 70B Chat | 97.1 |
| LLaMA 65B | 96.8 |
| Llama 2 34B | 96.7 |
| Llama 2 7B Chat | 96.5 |
| Llama 2 13B Chat | 96.5 |
| Code Llama Instruct 34B | 96.3 |
| Llama 2 70B | 96.2 |
| Falcon 7B | 96.1 |
| Llama 2 13B | 96.1 |
| LLaMA-13B | 95.7 |
| Falcon 40B | 95.2 |
| LLaMA 33B | 95.2 |
| LLaMA 7B | 94.9 |
| MPT-30B | 94.7 |
| Code Llama 34B | 93.9 |
| Code Llama Instruct 7B | 93.6 |
| Llama 2 7B | 93.0 |
| MPT-7B | 92.0 |
| Code Llama Instruct 13B | 91.9 |
| Code Llama 13B | 89.8 |
| Code Llama 7B | 86.7 |