OpenAI-Proof Q&A - pass@1
Code · 2025-08-07
OpenAI-Proof Q&A asks models to diagnose and explain 20 real research and engineering bottlenecks that each delayed an OpenAI project by at least a day, given a container with code access and run artifacts. Each solution is graded pass@1. Internal task set; diagnostic. Higher is better.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-5.1-Codex-Max | 8.0 |
| GPT-5.4 Mini | 7.5 |
| GPT-5.2 | 4.2 |
| GPT-5.4 | 4.2 |
| GPT-5 | 2.0 |
| GPT-5.5 | 1.7 |
| GPT-5.1 | 0.0 |