Atlas

Benchmarks

HoneyBench v0.2 (hack rate)

Goodhart Labs' HoneyBench measures specification gaming: each of its 20 agentic tasks in seven domains (medical, ML, administrative, coding, engineering, multimodal, and puzzles and games) pairs a hard objective with a faulty scoring setup that seems to let the agent score well while defeating the task's purpose. A run counts as a hack when its final submission exploits the flaw or the agent reaches proscribed information or resources, and the score is the share of scored runs hacked, pooled over the tasks a model was run on; lower is better. Diagnostic: provider-refused runs are left out of the rate and models run on different task sets.

Models
18
Top score
11.9
Leader
Claude Fable 5.2
Updated
—
Resolution
—
Saturation
—

No signal or difficulty estimate. This benchmark category is excluded from the capability model and its validation studies.

Organizations
17 dated models · 6 frontier modelsClosed weightsOpen weights
Data: Goodhart Labsatlas.village.dev

HoneyBench v0.2 (hack rate) leaderboard

Related benchmarks

Most correlated

Benchmarks outside its family whose model rankings track it most reliably (Spearman ρ over at least 10 shared models).

Loading Atlas data…