Benchmarks
347 sources · 1,258 models × 3,038 benchmarks · 158,912 scoresThe headline of UniPat's SPU paper and blog: each configuration's mean Spearman score over the public tasks it was scored on, adjusted for task difficulty by a model fitted jointly across all 13 configurations. Refusals and unrecoverable runs are dropped, so each configuration has its own denominator (30 to 100 tasks). Diagnostic; the primary `spu-v1-public-100` counts every run.
No signal or difficulty estimate. Held out of the fit by design.
| # | Model | Setting | Score | Source |
|---|---|---|---|---|
| 1 | Claude Opus 5.5 | max · Claude Code | 0.5 | |
| 2 | GPT-6 Astra | medium · Codex | 0.4 | |
| 3 | Claude Opus 5 | max · Claude Code | 0.4 | |
| 4 | Claude Fable 5.1 | max · Claude Code | 0.4 | |
| 5 | Claude Sonnet 5.5 | max · Claude Code | 0.4 | |
| 6 | GPT-6.1 Sol | max · Codex | 0.3 | |
| 7 | Kimi K3.1 Pro | max · Claude Code | 0.3 | |
| 8 | DeepSeek V4.1 Flash | max · Claude Code | 0.3 | |
| 9 | GLM-5.3 Flash | max · Claude Code | 0.2 | |
| 10 | Qwen3.8 Max (0902) | max · Claude Code | 0.2 |
Benchmarks outside its family whose model rankings track it most reliably (Spearman ρ over at least 10 shared models).