C3
General QA · 2019-04-21
C3 is a free-form multiple-choice Chinese machine-reading-comprehension benchmark drawn from Chinese-as-a-second-language exams, with dialogue and mixed-genre documents whose questions often require prior or commonsense knowledge. Questions have two to four options, so no single guessing floor is declared; scores are accuracy. Higher is better.
Top models (higher is better)
| Model | Score |
|---|---|
| Llama 3.1 405B | 79.7 |
| DeepSeek-V3-Base | 78.6 |
| Qwen1.5 72B | 78.2 |
| DeepSeek-V2 | 77.4 |
| DeepSeek LLM 67B Chat | 77.0 |
| Qwen2.5 72B | 76.7 |
| DeepSeek LLM 67B Base | 75.3 |
| Llama 3 70B | 74.0 |
| Mixtral 8x22B | 71.4 |
| DeepSeek LLM 7B Chat | 66.4 |
| DeepSeek LLM 7B Base | 65.4 |