RefSpatial-Bench
Multimodal · 2025-06-04
RefSpatial-Bench, from RoboRefer, evaluates spatial referring for robots: given a cluttered real scene and a multi-step spatial instruction, the model points to a target object (Location, 100 items) or a free-space placement (Placement, 100 items). A prediction succeeds when the point falls inside the ground-truth mask; the headline averages the Location and Placement success rates, and the 77-item Unseen split is reported separately.
Top models (higher is better)
| Model | Score |
|---|---|
| Qwen3.5 397B A17B | 73.6 |
| Qwen3.6 27B | 70.0 |
| Qwen3.5 122B A10B | 69.3 |
| Qwen3.5-27B | 67.7 |
| Gemini 3 Pro Preview | 65.5 |
| Qwen3.6 35B A3B | 64.3 |
| Qwen3.5 35B A3B | 63.5 |
| Qwen3.5-9B | 58.5 |
| Qwen3.5-4B | 54.6 |
| GPT-5.6 Luna | 52.5 |
| Gemma 4 12B IT | 51.7 |
| LFM3-VL-2B | 49.8 |
| Gemini 3.5 Flash-Lite | 33.6 |
| Qwen3.5-2B | 30.0 |
| Qwen3.5-0.8B | 23.5 |
| GPT-5 Nano | 12.6 |
| Gemini 2.5 Flash-Lite | 11.2 |
| GPT-5 Mini | 9.0 |
| Gemma 4 E2B IT | 6.0 |
| Gemma 4 31B IT | 4.7 |
| Sonnet 4.5 | 2.2 |