CMATH

Name: CMATH
Published: 2023-06-29

Math · 2023-06-29

CMATH is a Chinese elementary-school math word-problem benchmark of 1,698 problems from grades one to six, graded by final-answer match. Scores report accuracy; shot count is recorded per row. Higher is better.

Top models (higher is better)

Model	Score
DeepSeek-V4-Flash-Base	93.6
DeepSeek-V4-Pro-Base	90.9
Kimi K2 Base	90.3
DeepSeekMath 7B RL	88.8
Llama 4 Maverick	88.1
Kimi Linear 48B A3B Base	85.5
DeepSeekMath 7B Instruct	84.6
Qwen1.5 72B Chat	82.8
DeepSeek-V2-Chat	81.9
DeepSeek LLM 67B Chat	80.3
Llama 3 70B Instruct	79.2
DeepSeek-V2	78.7
Qwen1.5 72B	77.8
Llama 3.1 405B	77.3
Mixtral 8x22B Instruct	75.1
Llama 3 70B	73.9
Mixtral 8x22B	72.3
DeepSeekMath 7B Base	71.7
DeepSeek-V2-Lite-Chat	71.7
DeepSeekMoE 16B Chat	67.9
DeepSeek LLM 67B Base	63.0
DeepSeek-V2-Lite	58.4
Mistral 7B v0.1 Base	44.9
DeepSeekMoE 16B Base	40.4
DeepSeek LLM 7B Base	34.5