Atlas

Benchmarks

← All benchmarks

LiveBench 2024-08-31

General QA · 2024-08-31

The LiveBench release dated 2024-08-31, an earlier question set of the contamination-limited benchmark covering math, coding, reasoning, language, instruction following and data analysis. Scores are the overall category average for that release, kept apart from later LiveBench releases.

Top models (higher is better)

ModelScore
Qwen2.5-Max62.2
DeepSeek-V360.5
Claude 3.5 Sonnet (Oct 2024)60.3
Claude 3.5 Sonnet (June 2024)59.8
Qwen2.5-VL-72B-Instruct57.0
GPT-4o (2024-08-06)56.0
Qwen Plus (rolling alias)54.6
Llama 3.1 405B Instruct53.2
Qwen2.5 72B Instruct52.3
Qwen2.5 Instruct 32B50.7
Mistral Large 2 (Instruct 2407)48.5
DeepSeek-V2.547.3
Llama 3.1 70B Instruct46.6
Qwen2.5 14B Instruct 1M44.6
Qwen2.5 14B Instruct44.4
Qwen2.5-Turbo42.3
Qwen-Turbo (rolling alias)41.6
Qwen2 72B Instruct41.5
Gemma 2 27B IT39.6
Qwen2.5 7B Instruct35.9
Qwen2.5 7B Instruct 1M35.2
Qwen2-57B-A14B Instruct31.1
Gemma 2 9B IT30.6
Qwen2 7B Instruct29.2
Phi-3.5 Mini Instruct27.4
Qwen2.5 3B Instruct26.8
Llama 3.1 8B Instruct26.7
Gemma 2 2B IT20.1
Qwen2.5 1.5B Instruct18.8
Qwen2.5 0.5B Instruct12.6
Qwen2 1.5B Instruct12.4
Qwen2 0.5B Instruct7.4
Loading Atlas data…