SEED-Bench (image)
Multimodal · 2023-07-30
The nine image dimensions of SEED-Bench v1 (scene understanding, instance identity, attributes, location, counting, spatial relations, interaction, visual reasoning and text recognition): 14,233 four-option multiple-choice questions out of the benchmark's 19,242. Scores are accuracy over the image questions; the original protocol ranks options by likelihood while generative toolkits extract the chosen letter. SEED-Bench-2 is a separate benchmark.
Top models (higher is better)
| Model | Score |
|---|---|
| Gemma 4 12B IT | 80.6 |
| Gemini 3.5 Flash-Lite | 79.5 |
| Qwen3.5-4B | 78.9 |
| LFM3-VL-2B | 78.6 |
| LFM2.5-VL-3B | 77.7 |
| Qwen3.5-2B | 76.9 |
| LFM2-VL-3B | 76.5 |
| GPT-5.6 Luna | 76.2 |
| Gemma 4 E2B IT | 75.2 |
| LFM3-Omni-2B | 75.1 |
| InternVL3-2B | 75.0 |
| Qwen2.5-VL-3B-Instruct | 73.9 |
| Phi-4-multimodal-instruct | 72.2 |
| LFM2-VL-1.6B | 72.0 |
| LFM2-VL-450M | 63.6 |