OpenAI Professional Capture-the-Flag challenges - pass@12
Cybersecurity · 2025-12-18
OpenAI's Preparedness evaluation of professional-level capture-the-flag challenges: advanced, multi-step real-world security tasks solved in a Linux environment. The score is the share of challenges solved within 12 attempts (pass@12). The challenge set is not public; diagnostic. Higher is better.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-5.5 | 96.3 |
| GPT-5.4 | 88.2 |
| GPT-5.2 | 82.4 |
| GPT-5.4 Mini | 81.3 |
| GPT-5.1 | 43.0 |
| o3 | 27.7 |