Atlas

Benchmarks

← All benchmarks

DocVQA

Multimodal · 2020-07-01

DocVQA asks questions about scanned document images from the UCSF Industry Documents Library that require reading the text and understanding the layout. This row is the 5,188-question test split, scored on the official server by Average Normalized Levenshtein Similarity (ANLS), which gives partial credit to near-matching answers; whether external OCR text was supplied is noted per observation. Higher is better.

Top models (higher is better)

ModelScore
Qwen3-VL-235B-A22B-Instruct97.1
Qwen3 VL 32B Instruct96.9
Qwen2 VL 72B Instruct96.5
Qwen3 VL 235B A22B Thinking96.5
Qwen2.5-VL-72B-Instruct96.4
Qwen3-VL-8B-Instruct96.1
Qwen3 VL 32B Thinking96.1
Qwen2.5-VL-7B-Instruct95.7
Mistral Medium 395.3
Qwen3 VL 8B Thinking95.3
Qwen3 VL 4B Instruct95.3
Claude 3.5 Sonnet (June 2024)95.2
InternVL2.5-78B95.1
Qwen3 VL 30B A3B Instruct95.0
Mistral Small 3.2 24B Instruct 250694.9
Qwen2.5 VL 32B Instruct94.8
Nemotron Nano 12B V2 VL BF1694.7
Qwen2 VL 7B Instruct94.5
Llama 4 Scout Instruct94.4
Claude 3.5 Sonnet (Oct 2024)94.2
Qwen3 VL 4B Thinking94.2
Qwen2.5-VL-3B-Instruct93.9
Grok 293.6
Pixtral Large93.3
Qwen3 VL 2B Instruct93.3
Grok 2 Mini93.2
Gemini 1.5 Pro (May 2024)93.1
Gemini 1.5 Pro (source unspecified)93.1
Qwen VL Max (rolling alias)93.1
MiniCPM-o 2.693.0
Gemini 2.5 Flash93.0
Qwen3 VL 2B Thinking92.9
GPT-4o (2024-05-13)92.8
NVLM-D-72B92.6
Gemini 2.5 Pro92.6
Opus 4.192.5
Gemini 1.5 Pro 00292.3
Sonnet 492.0
GPT-591.5
Gemini 1.0 Ultra90.9
Loading Atlas data…