Atlas

Benchmarks

← All benchmarks

MM-MT-Bench

Multimodal · 2024-09-17

MM-MT-Bench is Mistral's open multimodal counterpart of MT-Bench: multi-turn instruction-following conversations over images (charts, documents, natural images, diagrams) whose answers an LLM judge (GPT-4o in Mistral's evaluation scripts) grades on a 1-10 scale against reference answers. Reported as the mean judge score. Higher is better.

Top models (higher is better)

ModelScore
Ministral 3 14B Instruct 25128.5
Ministral 3 8B Instruct 25128.1
Qwen3 VL 4B Instruct8.0
Qwen3-VL-8B-Instruct8.0
Ministral 3 3B Instruct 25127.8
GPT-4o (source unspecified)7.7
Claude 3.5 Sonnet (June 2024)7.5
Pixtral Large7.4
Mistral Small 3.1 24B Instruct 25037.3
Claude 3.5 Sonnet (Oct 2024)7.3
Gemma 3 27B IT7.0
Gemini 1.5 Pro 0026.8
Gemma 3 12B IT6.7
GPT-4o (2024-08-06)6.7
GPT-4o Mini6.6
Haiku 3.56.5
Pixtral 12B6.0
Llama 3.2 90B Vision Instruct5.5
Claude 3 Haiku5.5
Qwen2 VL 7B Instruct5.5
Gemma 3 4B IT5.2
Phi-3.5-vision-instruct4.5
Loading Atlas data…