ToxiGen - toxic generation rate
Safety · 2022-03-17
ToxiGen supplies implicitly toxic and benign statements about 13 minority groups. Used as a generation benchmark, the model continues prompts built from them and a fine-tuned RoBERTa toxicity classifier labels each continuation; the score is the percentage of continuations classified as toxic, so lower is better.
Top models (lower is better)
| Model | Score |
|---|---|
| Llama 2 7B Chat | 0.0 |
| Llama 2 13B Chat | 0.0 |
| Code Llama Instruct 34B | 0.0 |
| Code Llama Instruct 13B | 0.0 |
| Llama 2 70B Chat | 0.0 |
| Code Llama Instruct 7B | 0.0 |
| Falcon 7B | 14.5 |
| Code Llama 34B | 17.6 |
| Llama 2 34B | 21.2 |
| Llama 2 7B | 21.3 |
| LLaMA 65B | 21.8 |
| MPT-7B | 22.3 |
| Code Llama 13B | 22.4 |
| LLaMA 33B | 22.6 |
| MPT-30B | 22.6 |
| Code Llama 7B | 22.6 |
| LLaMA 7B | 23.0 |
| LLaMA-13B | 23.1 |
| Falcon 40B | 23.4 |
| Llama 2 70B | 24.6 |
| Llama 2 13B | 26.1 |