LingoQA
Multimodal · 2023-12-21
LingoQA is a video question-answering benchmark for autonomous driving with free-form questions about perception, reasoning and driving decisions in short driving clips. Answers are graded by the learned Lingo-Judge classifier, and scores report the share judged correct.
Top models (higher is better)
| Model | Score |
|---|---|
| Qwen3.8-Max | 84.8 |
| Qwen3.7-Plus (source unspecified) | 83.4 |
| Qwen3.5-27B | 82.0 |
| Qwen3.5 397B A17B | 81.6 |
| Qwen3.5 122B A10B | 80.8 |
| Qwen3.5-9B | 80.4 |
| Qwen3.5 35B A3B | 79.2 |
| Opus 4.5 | 78.8 |
| GPT-5.4 | 78.2 |
| Opus 4.6 | 77.6 |
| Qwen3.6 Plus (2026-04-02) | 76.0 |
| Qwen3.5-4B | 74.4 |
| Opus 4.8 | 73.8 |
| Gemini 3 Pro Preview | 72.8 |
| GPT-5.6 Sol | 72.6 |
| GPT-5.2 | 68.8 |
| Kimi K2.5 | 68.2 |
| Gemini 3.1 Pro Preview | 66.8 |
| GPT-5 Mini | 62.4 |
| GPT-5 Nano | 57.0 |
| Gemini 2.5 Flash-Lite | 17.8 |
| Sonnet 4.5 | 12.8 |