CMRC 2018
General QA · 2018-10-17
CMRC 2018 is a Chinese span-extraction reading-comprehension benchmark: each question about a Chinese Wikipedia passage is answered by a span from that passage. Base-model reports score it by exact match against the reference answers. Higher is better.
Top models (higher is better)
| Model | Score |
|---|---|
| DeepSeek-V3-Base | 76.3 |
| Llama 3.1 405B | 76.0 |
| Qwen2.5 72B | 75.8 |
| DeepSeek LLM 67B Base | 73.4 |
| Llama 3 70B | 73.3 |
| Mixtral 8x22B | 73.1 |
| Qwen1.5 72B | 66.6 |