QuAC
General QA · 2018-08-21
QuAC (Question Answering in Context) is a multi-turn, information-seeking question-answering benchmark in which a student asks free-form questions about a hidden Wikipedia section and a teacher answers with text spans, including context-dependent and unanswerable questions. Scores are word-level F1 averaged over the 7,354 development-set questions.
Top models (higher is better)
| Model | Score |
|---|---|
| Llama 3.1 405B | 53.6 |
| Llama 3 70B | 51.1 |
| Llama 3.1 70B | 51.1 |
| Llama 3.1 8B | 44.9 |
| Mixtral 8x22B | 44.9 |
| Llama 3 8B | 44.4 |
| Llama 3.2 3B | 42.9 |
| Gemma 7B | 42.4 |
| Llama 3.2 1B | 37.9 |