EvalPlus (average)
Code · 2023-05-02
The EvalPlus average reported by some base-model releases: the mean pass@1 over HumanEval, MBPP, HumanEval+ and MBPP+, where the plus variants re-score solutions against EvalPlus's much larger test suites. Because it averages four related code-generation scores, it is kept as its own measurement rather than pooled with any one of them. Higher is better.
Top models (higher is better)
| Model | Score |
|---|---|
| Kimi K2 Base | 80.3 |
| Qwen2.5 72B | 66.0 |
| DeepSeek-V3-Base | 65.6 |
| Llama 4 Maverick | 65.5 |
| Kimi Linear 48B A3B Base | 64.9 |
| Moonlight 16B A3B | 50.3 |