Atlas

Benchmarks

CyberBinGym Vulnerability Recovery

The share of CyberBinGym's 100 binary-only tasks on which the agent's submitted input triggers the expected sanitizer-detected failure in a hidden instrumented build of the vulnerable version while the patched build stays clean: CyberGym's differential proof-of-concept test, here from stripped binaries alone. It is scored independently of exploitation, since an accepted exploit may use a different bug and need not pass this test. Higher is better.

Models
3
Top score
88.0
Leader
GPT-5.6 Sol
Updated
Nov 7, 2026
Resolution
—
Saturation
—

No signal or difficulty estimate. Too few models have been measured on this benchmark to estimate its difficulty.

Organizations
3 dated models · 1 frontier modelsClosed weightsOpen weights
Data: CyberBinGym Authorsatlas.village.dev

CyberBinGym Vulnerability Recovery leaderboard

#ModelSettingScoreSource
1GPT-5.6 Solnot reported · CyberBinGym Inspect sandbox (static and dynamic analysis containers, a victim service, proxy-restricted egress); Codex CLI with web search disabled; nominal 1,000 LLM-call budget; trusted-access program with safeguards disabled
88.0
2Claude Opus 5not reported · CyberBinGym Inspect sandbox (static and dynamic analysis containers, a victim service, proxy-restricted egress); Claude Code with its Bash and WebSearch tools replaced by a bridged shell; nominal 1,000 LLM-call budget; trusted-access program with safeguards disabled
85.0
3Kimi K3not reported · CyberBinGym Inspect sandbox (static and dynamic analysis containers, a victim service, proxy-restricted egress); Kimi Code with WebSearch and FetchURL disabled; nominal 1,000 LLM-call budget; trusted-access program with safeguards disabled
64.0
Loading Atlas data…