Atlas

Benchmarks

← All benchmarks

EgoSchema

Multimodal · 2023-08-17

EgoSchema is a very-long-form video question-answering benchmark built on Ego4D: each human-curated five-way multiple-choice question covers a three-minute egocentric clip and targets long temporal understanding. Rows use accuracy on the full test set, scored through the official evaluation server, unless a note says otherwise; higher is better.

Top models (higher is better)

ModelScore
Qwen2 VL 72B Instruct77.9
Qwen2.5-VL-72B-Instruct76.2
Gemini 1.5 Pro (May 2024)72.2
GPT-4o (2024-05-13)72.2
Gemini 2.0 Pro Experimental 02-0571.9
Gemini 2.0 Flash Experimental71.5
Gemini 1.5 Pro 00271.2
Gemini 2.0 Flash71.1
Gemini 2.0 Flash-Lite Preview 02-0567.2
Gemini 1.5 Flash 00266.8
Qwen2 VL 7B Instruct66.7
Gemini 1.5 Flash (May 2024 report)65.7
Qwen2.5-VL-7B-Instruct65.0
Qwen2.5-VL-3B-Instruct64.8
GPT-4 Turbo63.9
Gemini 1.5 Pro (February 2024 report)63.2
Qwen2 VL 2B Instruct54.9
Loading Atlas data…