Atlas

Benchmarks

PMPP-Hard

PMPP-Hard (v0.1.1, released on GitHub by the developer Sinatras and packaged in Prime Intellect's RL Residency environments) asks a coding agent to write and optimize 69 GPU kernels, 60 in CUDA, 8 in Triton and 1 in CUTLASS, for NVIDIA Blackwell sm_120 under Verifiers 0.2.0. The agent never sees reference implementations or authoritative graders; only its submitted source is copied into a clean scoring sandbox, where a task passes when it builds, passes the authoritative correctness test and, on the 48 performance-gated tasks, runs within 1.25 times the reference's median time over five interleaved timing pairs with matching output digests, under KernelGuard source and timing checks. The score is the share of the 69 tasks passed with one accepted rollout per task; the release campaign's harness, reasoning setting and rollout limit (1,500 or 4,500 seconds) vary by model.

Models
10
Top score
71.0
Leader
Kimi K3
Updated
Aug 3, 2026
Resolution
~3 mo
Sharp
Saturation
~Mar 2027
Projected
Signal
58thpercentile
195 benchmarks from 0.8 to 3.5224 benchmarks from 3.5 to 6.1133 benchmarks from 6.1 to 8.898 benchmarks from 8.8 to 11.566 benchmarks from 11.5 to 14.126 benchmarks from 14.1 to 16.815 benchmarks from 16.8 to 19.59 benchmarks from 19.5 to 22.18 benchmarks from 22.1 to 24.84 benchmarks from 24.8 to 27.55 benchmarks from 27.5 to 30.12 benchmarks from 30.1 to 32.84 benchmarks from 32.8 to 35.45 benchmarks from 35.4 to 38.11 benchmark from 38.1 to 40.82 benchmarks from 43.4 to 46.11 benchmark from 62.1 to 64.8

333rd strongest of 798

Signal-to-noise +7.07

Difficulty
66thpercentile
1 benchmark from 5.5 to 14.41 benchmark from 14.4 to 23.22 benchmarks from 32.1 to 41.01 benchmark from 41.0 to 49.82 benchmarks from 49.8 to 58.79 benchmarks from 58.7 to 67.67 benchmarks from 67.6 to 76.418 benchmarks from 76.4 to 85.324 benchmarks from 85.3 to 94.241 benchmarks from 94.2 to 103.031 benchmarks from 103.0 to 111.939 benchmarks from 111.9 to 120.850 benchmarks from 120.8 to 129.676 benchmarks from 129.6 to 138.560 benchmarks from 138.5 to 147.4118 benchmarks from 147.4 to 156.2124 benchmarks from 156.2 to 165.1120 benchmarks from 165.1 to 174.039 benchmarks from 174.0 to 182.820 benchmarks from 182.8 to 191.710 benchmarks from 191.7 to 200.61 benchmark from 200.6 to 209.44 benchmarks from 209.4 to 218.3

272nd hardest of 798

Capability needed 159.1

Organizations
10 dated models · 3 frontier modelsClosed weightsOpen weights
Data: PMPP-Hard Authorsatlas.village.dev

PMPP-Hard leaderboard

Related benchmarks

Most correlated

Benchmarks outside its family whose model rankings track it most reliably (Spearman ρ over at least 10 shared models).

Loading Atlas data…