AI2D (test)

Name: AI2D (test)
Published: 2016-03-24

Multimodal · 2016-03-24

AI2 Diagrams: 3,088 four-option test questions on grade-school science diagrams, scored as multiple-choice accuracy. This row holds test-set accuracy as reported by model developers under their own prompts; protocols differ on whether the lettered answer regions in a diagram are masked, which observations note. It is a component of the fixed-harness OpenVLM family (ai2d-vlmevalkit). Higher is better.

Top models (higher is better)

Model	Score
Claude 3.5 Sonnet (Oct 2024)	95.3
Claude 3.5 Sonnet (June 2024)	94.7
Gemini 1.5 Pro (May 2024)	94.4
Qwen3.6 Plus (2026-04-02)	94.4
GPT-4o (2024-05-13)	94.2
Gemini 3 Pro Preview	94.1
Qwen3.5 397B A17B	93.9
Mistral Small 3.1 24B Instruct 2503	93.7
Mistral Medium 3	93.7
GPT-4o (source unspecified)	93.3
Qwen3.5 122B A10B	93.3
Mistral Small 3.2 24B Instruct 2506	92.9
Qwen3.5-27B	92.9
Qwen3.6 35B A3B	92.7
Qwen3.5 35B A3B	92.6
Llama 3.2 90B Vision Instruct	92.3
GPT-5.2	92.2
Haiku 3.5	92.1
Qwen3.5 Plus (2026-02-15)	92.1
Qwen3.5-Omni-Plus	91.2
Llama 3.2 11B Vision Instruct	91.1
Gemini 2.5 Pro	90.9
Kimi K2.5	90.8
Qwen3.5-9B	90.2
Qwen3-VL-235B-A22B-Instruct	89.7
GPT-5	89.7
Qwen3.5-4B	89.6
Qwen3 VL 32B Instruct	89.5
GPT-4 Turbo	89.4
Qwen3 VL 235B A22B Thinking	89.2
InternVL2.5-78B	89.1
Qwen3.5-Omni-Flash	89.0
Gemma 4 31B IT	89.0
Qwen3 VL 32B Thinking	88.9
Claude 3 Sonnet	88.7
Grok 1.5V	88.3
Gemma 4 26B A4B IT	88.3
GPT-5 Mini	88.2
Opus 3	88.1
GPT-4o Mini	88.1