We-Math (testmini, strict score)
Multimodal · 2024-07-01
We-Math's 1,740-question testmini split decomposes 525 composite visual math problems into one-step sub-problems over hierarchical knowledge concepts, and classifies each composite problem as insufficient knowledge, inadequate generalization (IG), complete mastery (CM) or rote memorization. The strict score is (CM + 0.5 x IG) / 525, where CM requires every sub-problem and the composite answer to be right; the loose score counts more problems as mastered.
Top models (higher is better)
| Model | Score |
|---|---|
| Qwen3.5-4B | 81.0 |
| Gemma 4 12B IT | 78.2 |
| GPT-5.6 Luna | 70.3 |
| LFM3-VL-2B | 68.6 |
| Qwen3.5-2B | 59.8 |
| Gemini 3.5 Flash-Lite | 59.1 |
| Gemma 4 E2B IT | 44.2 |