ProgramBench
Code · 2026-05-05
Reverse-engineering benchmark in which an agent is given only a compiled executable and its documentation and must re-implement the program without access to any of its source. Scored across 200 tasks, from small utilities to projects the size of FFmpeg and SQLite, against more than 248,000 hidden behavioral tests. There is no partial credit: a task resolves only if every test for it passes.