SAGE v1.1
Math · 2026-10-27
SAGE v1.1 (Student Assessment with Generative Evaluation), Vals' October 27, 2026 revision of SAGE. Models read scans of handwritten undergraduate work and a rubric of possible point deductions and say which deductions apply; a GPT-5 judge checks each rubric item against the answer key. The score is class-balanced check accuracy, the mean of the share of applied deductions caught and the share of non-applied deductions correctly withheld, so never or always deducting scores 50%. Overall weights the three subjects PDEs 20%, Linear Algebra 30% and Computer Science 50%; standard errors bootstrap whole submissions. v1 scored the share of submissions graded entirely correctly and had Automata and Calculus subjects, so v1.1 is not comparable with sage.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Opus 5.5 | 91.3 |
| GPT-6.1 Sol | 90.5 |
| Claude Haiku 5.5 | 89.8 |
| Gemini 4 Argon | 89.5 |
| GLM-5.3 Flash | 89.3 |
| Qwen3.7-Plus (source unspecified) | 89.1 |
| Grok 4.7 | 88.7 |
| Gemini 3.8 Flash | 88.2 |
| Claude Sonnet 5.5 | 88.0 |
| GPT-5.6 Terra | 87.0 |
| Kimi K3 | 86.8 |
| Ember-1 | 86.4 |
| Muse Spark Open | 85.8 |
| MiniMax M3 | 85.5 |
| Qwen3.5 397B A17B | 85.5 |
| DeepSeek V4.1 Flash | 85.4 |
| MiniMax M3.1 Flash | 85.3 |
| Qwen3.5-Flash (source unspecified) | 84.9 |
| MiMo-V2.6-Pro | 84.4 |
| GPT-6 Luna | 83.4 |
| MiMo-V2.6-Flash | 83.3 |
| Gemini 3.1 Pro Preview | 83.2 |
| Gemma 4 31B IT | 82.8 |
| Gemini 3.5 Flash-Lite | 79.5 |
| Grok 4.1 Fast | 74.5 |
| Mistral Medium 3.5 | 72.0 |
| Mistral Small 4 | 59.2 |