Atlas

Benchmarks

PMPP-Hard (Thinking Machines 10-hour AUC)

Thinking Machines Lab's adaptation of PMPP-Hard's 69 GPU kernel tasks, reported with Inkling 2 Small's launch. Each model runs at its maximum available reasoning setting in the Pi harness with a shared context-compaction policy, four independent attempts per task and a 10-hour budget; Thinking Machines fixed task and grading issues it found, and each submitted solution gets two independent LLM audits, with a third review on disagreement. The score is the area under the mean tasks-solved curve over 0 to 10 hours, in task-hours (maximum 690), so it rewards solving early as well as solving at all. Diagnostic: the fixes and audits are unpublished and the protocol differs from the v0.1.1 release's.

Models
1
Top score
494
Leader
Inkling 2 Small
Updated
Nov 5, 2026
Resolution
—
Saturation
—

No signal or difficulty estimate. Held out of the fit by design.

Organizations

Not enough dated models to plot (1 of 1 have a release date).

PMPP-Hard (Thinking Machines 10-hour AUC) leaderboard

#ModelSettingScoreSource
1Inkling 2 Small0.99 · pi
494
Loading Atlas data…