LiveCodeBench
Code · 2024-03-12
LiveCodeBench evaluates code generation on newly released competitive-programming problems to limit contamination. Scores report pass@1 accuracy in percentage points.
Top models (higher is better)
| Model | Score |
|---|---|
| Qwen3.5 122B A10B | 74.0 |
| Qwen3-Next-80B-A3B-Thinking | 66.0 |
| DeepSeek-R1 | 65.9 |
| Mistral Small 4 | 63.6 |
| Magistral Medium 1.1 | 59.4 |
| Magistral Small 1.1 | 59.2 |
| DeepSeek-V3-0324 | 49.2 |
| GPT-4.5 | 44.4 |
| Claude 3.7 Sonnet | 42.2 |
| Qwen3 Next 80B A3B Instruct | 40.0 |
| Codestral 25.01 | 37.9 |
| Mistral Large 3 675B Base 2512 | 34.4 |
| Codestral 2405 | 31.5 |
| Mistral Medium 3 | 29.1 |
| Code Llama Instruct 70B | 20.0 |