Atlas

Benchmarks

← All benchmarks

LiveCodeBench - August 2024 to February 2025

Code

LiveCodeBench code-generation problems in the August 1, 2024 to February 1, 2025 date window, labelled "24.08-25.02" by Qwen, "2024-08 to 2025-01" in the DeepSeek-R1 report and 20240801-20250201 or 2408-2502 by NVIDIA. This exact historical problem cohort is retained as a diagnostic rather than pooled with other LiveCodeBench windows.

Top models (higher is better)

ModelScore
Nemotron Cascade 14B Thinking77.5
Nemotron Cascade 8B Thinking74.5
Nemotron Cascade 8B74.3
Nemotron Nano 9B V268.2
DeepSeek-R165.9
Qwen3 14B65.2
QwQ-32B63.4
Qwen3 8B61.2
DeepSeek-R1-Distill-Llama-70B57.5
DeepSeek-R1-Distill-Qwen-32B57.2
o1-mini53.8
DeepSeek R1 Distill Qwen 14B53.1
DeepSeek-R1-Zero50.0
Llama 3.1 Nemotron Nano 8B V146.6
Llama 3.3 Nemotron Super 49B V145.5
QwQ-32B-Preview41.9
DeepSeek R1 Distill Llama 8B39.6
DeepSeek-R1-Distill-Qwen-7B37.6
DeepSeek-V336.2
DeepSeek-R1-Distill-Qwen-1.5B16.9
Llama 3.1 8B Instruct11.8
Loading Atlas data…