← All benchmarks
DecisionPivot core pair accuracy
General QA · 2026-10-20
DecisionPivot's core track: 7,800 generated items in 13 task families testing routing, guardrail and verification decisions, grouped so that a contrast group counts only when every variant in it is right. Macro-averaged over families. Diagnostic, because the protocol differs by model.
Benchmark source: arXiv
Top models (higher is better)
| Model | Score | Source |
|---|
| GLM-5.3 | 82.2 | DecisionPivot paper Table D10 (GLM-5.3, lowest reasoning effort, pair accuracy) · StartLux |
| Gemma 4 31B IT | 37.2 | DecisionPivot paper Table D10 (Gemma-4-31B-it, pair accuracy) · StartLux |
| Gemma 4 26B A4B IT | 20.4 | DecisionPivot paper Table D10 (Gemma-4-26B-A4B-it, pair accuracy) · StartLux |
| Qwen3.8 27B | 12.9 | DecisionPivot paper Table D10 (Qwen3.8-27B, pair accuracy) · StartLux |
| Qwen3.5-9B | 11.4 | DecisionPivot paper Table D10 (Qwen3.5-9B, pair accuracy) · StartLux |
| Qwen3.6 35B A3B | 9.9 | DecisionPivot paper Table D10 (Qwen3.6-35B-A3B, pair accuracy) · StartLux |
| Qwen3.5-4B | 9.6 | DecisionPivot paper Table D10 (Qwen3.5-4B, pair accuracy) · StartLux |
| Gemma 3 12B IT | 8.5 | DecisionPivot paper Table D10 (Gemma-3-12B-it, pair accuracy) · StartLux |
| Phi-4 | 8.2 | DecisionPivot paper Table D10 (Phi-4, pair accuracy) · StartLux |
| Granite 3.3 8B Instruct | 4.8 | DecisionPivot paper Table D10 (Granite-3.3-8B-Instruct, pair accuracy) · StartLux |
| Qwen3.5-2B | 3.6 | DecisionPivot paper Table D10 (Qwen3.5-2B, pair accuracy) · StartLux |
| Qwen3.5-0.8B | 1.5 | DecisionPivot paper Table D10 (Qwen3.5-0.8B, pair accuracy) · StartLux |