Atlas

Benchmarks

← All benchmarks

VQAv2 (validation)

Multimodal · 2016-12-02

VQAv2 open-ended visual question answering on natural images, scored with the standard VQA-match accuracy (an answer counts fully when at least three of ten annotators gave it) on the validation split. Higher is better.

Top models (higher is better)

ModelScore
Pixtral Large80.9
Pixtral 12B78.6
GPT-4o (source unspecified)77.8
GPT-4o (2024-08-06)76.4
Qwen2 VL 7B Instruct75.9
Claude 3.5 Sonnet (June 2024)70.7
Gemini 1.5 Pro 00270.6
Claude 3.5 Sonnet (Oct 2024)69.5
Claude 3 Haiku68.4
Llama 3.2 90B Vision Instruct67.0
Phi-3.5-vision-instruct56.1
Loading Atlas data…