Chinese SimpleQA
General QA · 2024-11-11
Chinese SimpleQA is a short-form factuality benchmark of 3,000 Chinese questions across six major topics, each with a single verifiable answer. A judge model grades every answer correct, incorrect or not attempted; this row is the share graded correct. Higher is better.
Top models (higher is better)
| Model | Score |
|---|---|
| Gemini 3.1 Pro Preview | 85.9 |
| DeepSeek-V4-Pro | 84.4 |
| DeepSeek-V4-Flash | 78.9 |
| Kimi K2 Base | 77.6 |
| GPT-5.4 | 76.8 |
| Opus 4.6 | 76.4 |
| Kimi K2.6 | 75.9 |
| GLM-5.1 | 75.0 |
| DeepSeek-V3-Base | 72.1 |
| DeepSeek-R1-Zero | 66.4 |
| DeepSeek-R1 | 63.7 |
| DeepSeek-V2.5 | 54.1 |
| Kimi Linear 48B A3B Base | 53.5 |
| Llama 4 Maverick | 53.5 |
| Qwen2.5 72B | 50.5 |
| Llama 3.1 405B Instruct | 50.4 |
| DeepSeek-V2-Chat | 48.5 |
| Qwen2.5 72B Instruct | 48.4 |
| o1-mini | 40.3 |
| Moonlight 16B A3B | 34.7 |