← All benchmarks
PLE Hidden Test Set
Professional Work · 2026-10-21
PLE's 56 hidden test tasks (18 electrical, 9 civil, 23 mechanical, 6 structural), the default view of the authors' leaderboard: the mean task score in percent over the same runs as the 85-task ple score, scored the same way. The other 29 tasks are public as the dev set, so this split is the part of PLE a harness can't have seen.
Benchmark source: Ple Bench
Top models (higher is better)
| Model | Score | Source |
|---|
| Claude Opus 5.5 | 48.6 | PLE leaderboard, 56-task view (Claude Opus 5.5 (OH)) · Google |
| GPT-6 Astra | 45.8 | PLE leaderboard, 56-task view (GPT 6 Astra (Cdx)) · Google |
| Claude Sonnet 5.5 | 45.1 | PLE leaderboard, 56-task view (Claude Sonnet 5.5 (Hrm)) · Google |
| Claude Opus 5 | 42.6 | PLE leaderboard, 56-task view (Claude Opus 5 (OH)) · Google |
| Gemini 4 Argon | 41.1 | PLE leaderboard, 56-task view (Gemini 4 Argon (CC)) · Google |
| Claude Fable 5 | 34.5 | PLE leaderboard, 56-task view (Claude Fable 5 (CC)) · Google |
| Grok 4.7 | 30.2 | PLE leaderboard, 56-task view (Grok 4.7 (Pi)) · Google |
| Gemini 3.8 Flash | 28.2 | PLE leaderboard, 56-task view (Gemini 3.8 Flash (OH)) · Google |
| GLM-5.2 | 25.0 | PLE leaderboard, 56-task view (GLM 5.2 (Cdx)) · Google |
| Gemini 3.1 Pro Preview | 14.8 | PLE leaderboard, 56-task view (Gemini 3.1 Pro (Hrm)) · Google |
| Kimi K2 Thinking | 6.2 | PLE leaderboard, 56-task view (Kimi K2 Thinking (OC)) · Google |
| Qwen3-Coder-480B-A35B-Instruct | 6.2 | PLE leaderboard, 56-task view (Qwen3 Coder 480B (CC)) · Google |
| MiniMax M2 | 5.7 | PLE leaderboard, 56-task view (MiniMax M2 (CC)) · Google |
| gpt-oss-120b | 4.6 | PLE leaderboard, 56-task view (gpt-oss 120B (OH)) · Google |
| Gemma 4 26B A4B IT | 4.5 | PLE leaderboard, 56-task view (Gemma 4 26B (OH)) · Google |
| Qwen3 Next 80B A3B Instruct | 4.3 | PLE leaderboard, 56-task view (Qwen3-Next 80B (OH)) · Google |
| Llama 4 Maverick Instruct | 0.7 | PLE leaderboard, 56-task view (Llama 4 Maverick (OH)) · Google |