Atlas

Benchmarks

← All benchmarks

MathVerse (testmini)

Multimodal · 2024-03-21

MathVerse tests whether multimodal models truly read diagrams in math problems, rendering each of its testmini problems in five versions that move information from the text into the image. This row is the overall testmini accuracy as reported by model developers, who seldom state which problem versions were averaged.

Top models (higher is better)

ModelScore
Qwen3 VL 235B A22B Thinking85.0
GPT-584.1
Gemini 2.5 Pro82.9
Qwen3 VL 32B Thinking82.6
Qwen3 VL 30B A3B Thinking79.6
GPT-5 Mini78.8
Gemini 2.5 Flash77.7
Qwen3 VL 8B Thinking77.7
Qwen3 VL 32B Instruct76.8
Qwen3 VL 4B Thinking75.2
GPT-5 Nano74.2
Qwen3-VL-235B-A22B-Instruct72.5
Opus 4.170.6
Qwen3 VL 30B A3B Instruct70.2
Sonnet 468.6
Qwen3 VL 2B Thinking66.9
Qwen3-VL-8B-Instruct62.1
Qwen2.5-VL-72B-Instruct57.6
Qwen3 VL 2B Instruct52.1
InternVL2.5-78B51.7
GPT-4o (2024-05-13)50.2
Qwen2.5-VL-7B-Instruct49.2
Qwen2.5-VL-3B-Instruct47.6
Qwen3 VL 4B Instruct46.8
Loading Atlas data…