MotionBench (dev)
Multimodal · 2025-01-06
MotionBench evaluates fine-grained video motion understanding with about 8,050 multiple-choice questions over 5,385 videos in six motion-oriented categories; roughly half form the public dev split with answers and the rest a hidden test split. Scores are accuracy, and the paper puts random guessing at 25 percent.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-5.6 Luna | 67.7 |
| LFM3-VL-2B | 64.0 |
| Qwen3.5-4B | 61.8 |
| Gemini 3.5 Flash-Lite | 61.0 |
| Qwen3.5-2B | 56.1 |
| Gemma 4 12B IT | 53.9 |
| Gemma 4 E2B IT | 41.5 |