Shroombench (mushroom identification, top-1)
Multimodal · 2026-09-02
Quesma's mushroom-identification benchmark (repository stared/mushroom-hunting-llm-bench): 1,040 photos of 55 edible and poisonous species found in Poland, up to 20 per species, from the FungiTastic test split (Atlas of Danish Fungi, expert-verified labels) topped up from the validation split. A vision model sees one photo with a prompt asking for a JSON array of the five most likely species as Latin binomials, with no tools. The score is the share of photos whose first name matches the expert label, by string matching with accepted synonyms; empty or token-exhausted answers count as wrong.
Top models (higher is better)
| Model | Score |
|---|---|
| Gemini 3.8 Flash | 65.0 |
| Gemini 3.6 Flash | 64.0 |
| Gemini 3.7 Flash | 61.0 |
| GPT-6 Astra | 54.0 |
| Claude Fable 5 | 53.0 |
| Kimi K3 | 53.0 |
| Claude Fable 5.1 | 51.0 |
| Muse Spark 1.3 | 49.0 |
| GLM-5.3 Flash | 47.0 |
| Claude Opus 5 | 44.0 |
| Grok 4.6 | 44.0 |
| GPT-5.6 Sol | 42.0 |
| GPT-5.6 Sol Pro | 39.0 |
| Muse Spark 1.2 | 39.0 |
| GPT-5.6 Terra | 35.0 |
| Sonnet 5 | 32.0 |
| DeepSeek V4 Flash Vision Exp | 30.0 |
| GPT-5.6 Luna | 30.0 |
| Qwen3.8 Flash | 27.0 |
| MiniMax M3 | 24.0 |
| Qwen3.8 27B | 13.0 |