LiveCodeBench - July to September 2024

Code

LiveCodeBench code-generation problems in the July 2024 to September 2024 date window that the Qwen2.5-Coder report labels "LiveCodeBench (2407-2409)". This exact historical problem cohort is retained as a diagnostic rather than pooled with other LiveCodeBench windows.

Top models (higher is better)

Model	Score
o1-mini	60.0
o1 Preview	43.1
GPT-4o (2024-08-06)	34.6
Claude 3.5 Sonnet (June 2024)	32.1
Claude 3.5 Sonnet (Oct 2024)	31.6
Qwen2.5 Coder 32B Instruct	31.4
GPT-4o Mini	28.3
Qwen2.5 Coder 14B Instruct	23.4
Codestral 2405	22.6
Qwen2.5-Coder-7B-Instruct	18.2
DeepSeek-Coder-V2-Lite-Instruct	16.3
Qwen2.5 Coder 3B Instruct	10.8
Code Llama Instruct 34B	8.4
CodeQwen1.5 7B Chat	7.9
Qwen2.5 Coder 1.5B Instruct	6.1
Code Llama Instruct 70B	3.3
Qwen2.5 Coder 0.5B Instruct	2.0