ChID
General QA · 2019-06-04
ChID is a Chinese idiom cloze benchmark: blanks in a passage must be filled with the correct four-character idiom from a candidate list that includes near-synonym distractors. Scores are accuracy; candidate-list size varies by setting, so no guessing floor is declared. Higher is better.
Top models (higher is better)
| Model | Score |
|---|---|
| DeepSeek-V2 | 92.7 |
| DeepSeek LLM 67B Base | 92.1 |
| DeepSeekMoE 16B Base | 89.4 |
| DeepSeek LLM 7B Base | 89.3 |
| Llama 3 70B | 83.2 |
| DeepSeek LLM 67B Chat | 72.6 |
| DeepSeek LLM 7B Chat | 64.9 |
| Mixtral 8x22B | 57.0 |