POPE (F1, pooled over three settings)
Multimodal · 2023-05-17
POPE measures object hallucination with yes/no 'Is there a <object> in the image?' questions on 500 COCO images, sampling negative objects at random, by frequency (popular) or by co-occurrence (adversarial): 3,000 questions per setting, 9,000 in all. This row is F1 with 'yes' as the positive class computed once over all 9,000 pooled answers, which differs slightly from the mean of the three per-setting F1 scores the paper reports.
Top models (higher is better)
| Model | Score |
|---|---|
| InternVL3-2B | 90.1 |
| LFM3-Omni-2B | 89.3 |
| Gemini 3.5 Flash-Lite | 89.2 |
| LFM2-VL-3B | 89.0 |
| LFM3-VL-2B | 88.7 |
| LFM2.5-VL-3B | 88.7 |
| Qwen3.5-2B | 88.1 |
| Qwen3.5-4B | 88.1 |
| Gemma 4 12B IT | 87.6 |
| LFM2.5-VL-450M | 86.9 |
| Qwen2.5-VL-3B-Instruct | 86.2 |
| GPT-5.6 Luna | 85.0 |
| Gemma 4 E2B IT | 84.2 |
| LFM2-VL-450M | 83.8 |
| Phi-4-multimodal-instruct | 83.3 |