MM-IFEval
Multimodal · 2025-04-10
MM-IFEval, released with MM-IFEngine, tests multimodal instruction following with 400 image-grounded prompts: 300 compose-level items carrying several output constraints and 100 perception-level items with reference answers. Compose-level items score the mean of their constraints (checked by rules or an LLM judge), perception-level items are judged against the reference, and the overall is the item-weighted mean; the canonical judge is GPT-4o.
Top models (higher is better)
| Model | Score |
|---|---|
| Gemma 4 12B IT | 83.0 |
| GPT-5.6 Luna | 82.0 |
| Qwen3.5-4B | 79.5 |
| Gemini 3.5 Flash-Lite | 73.2 |
| LFM3-VL-2B | 72.8 |
| Qwen3.5-2B | 62.0 |
| Gemma 4 E2B IT | 61.6 |
| LFM2.5-VL-3B | 60.6 |
| LFM2-VL-3B | 51.8 |
| LFM2-VL-1.6B | 46.4 |
| LFM2.5-VL-450M | 45.0 |
| Qwen2.5-VL-3B-Instruct | 38.6 |
| InternVL3-2B | 38.5 |
| LFM2-VL-450M | 33.1 |