PI-AutomationBench (strict)
Agents · 2026-11-03
The strict score the Prime Intellect post reports beside PI-AutomationBench's partial score, on the same runs in 'mode: Zapier'. It is its own benchmark because it scores the same tasks by a different rule, which the post recommends reporting since partial scoring is close to saturation. The post doesn't define strict further or state the task count.
Benchmark source: Primeintellect
Top models (higher is better)
| Model | Score | Source |
|---|---|---|
| Claude Opus 5.5 | 70.8 | Prime Intellect, Learnings from Auditing Public Evaluations · Prime Intellect |
| DeepSeek V4.1 Flash 1017 | 69.5 | Prime Intellect, Learnings from Auditing Public Evaluations · Prime Intellect |