EmbSpatial-Bench
Multimodal · 2024-06-09
EmbSpatial-Bench tests egocentric spatial understanding for embodied agents with 3,640 four-option multiple-choice questions over 2,181 images from Matterport3D, AI2-THOR and ScanNet, covering six spatial relations (above, below, left, right, close and far). Scores are accuracy.
Top models (higher is better)
| Model | Score |
|---|---|
| Qwen3.5-Omni-Plus | 85.4 |
| Qwen3.6 27B | 84.6 |
| Qwen3.5 397B A17B | 84.5 |
| Qwen3.5-27B | 84.5 |
| Qwen3.6 35B A3B | 84.3 |
| Qwen3.5 122B A10B | 83.9 |
| Qwen3.5 Plus (2026-02-15) | 83.4 |
| Qwen3.5 35B A3B | 83.1 |
| Qwen3.5-9B | 83.0 |
| LFM3-VL-2B | 82.9 |
| Qwen3.5-Omni-Flash | 82.7 |
| Qwen3.5-4B | 82.4 |
| GPT-5.2 | 81.3 |
| GPT-5 Mini | 80.7 |
| Gemma 4 12B IT | 80.2 |
| Kimi K2.5 | 77.4 |
| Opus 4.5 | 75.7 |
| GPT-5.6 Luna | 75.5 |
| Gemini 3.5 Flash-Lite | 75.3 |
| GPT-5 Nano | 74.2 |
| Sonnet 4.5 | 71.8 |
| LFM3-Omni-2B | 70.5 |
| Qwen3.5-0.8B | 68.6 |
| Gemma 4 E2B IT | 67.9 |
| Qwen3.5-2B | 66.4 |
| Gemini 2.5 Flash-Lite | 66.1 |
| Gemini 3 Pro Preview | 61.2 |
| Phi-4-multimodal-instruct | 57.8 |