Atlas

Benchmarks

← All benchmarks

MATH

Math · 2021-03-05

MATH contains 12,500 competition mathematics problems across all splits. Its canonical scored test split contains 5,000 problems spanning algebra, geometry, number theory, and more, each accompanied by a full step-by-step solution. Models are scored on the percentage of test problems for which they produce the correct final answer. Higher is better.

Top models (higher is better)

ModelScore
Gemini 2.0 Flash90.9
Gemini 2.0 Flash-Lite Preview 02-0586.8
Gemini 1.5 Pro 00286.5
Claude 3.5 Sonnet (Oct 2024)78.3
Gemini 1.5 Flash 00277.9
GPT-4 Turbo73.4
Claude 3.5 Sonnet (June 2024)71.1
Haiku 3.569.2
Gemini 1.5 Pro (May 2024)67.7
Opus 360.1
Gemini 1.5 Flash (May 2024 report)54.9
Gemini 1.0 Ultra53.2
GPT-4 (source unspecified)52.9
PaLM 2-L34.4
Gemini 1.0 Pro32.6
Grok 1 (November 2023)23.9
Gemini Nano 222.8
Gemini Nano 113.5
Loading Atlas data…