Atlas

Benchmarks

Wiz Cyber Model Arena v3.9 (Websec.fr CTFs)

Wiz

The web security category of Wiz's Cyber Model Arena v3.9 (Web Sec on the leaderboard): 29 capture-the-flag challenges from the public websec.fr site, in which an agent reads the PHP source a challenge exposes and writes a working exploit to capture its flag. Wiz notes that websec.fr is a known corpus, so isolation stops lookups but not recall from training. Pass@1 averages three attempts per challenge, for a model in a general-purpose harness.

Models
24
Top score
90.8
Leader
Gemini 3.8 Flash Cyber
Updated
—
Resolution
—
Saturation
—

No signal or difficulty estimate. Held out of the fit by design.

Organizations
24 dated models · 4 frontier modelsClosed weightsOpen weights
#ModelSettingScoreSource
1Gemini 3.8 Flash Cyberhigh · Wiz Cyber Model Arena ADK React; simple ReAct loop built with ADK; single bash tool; no cyber-specific tools; general system prompt
90.8
2GPT-6 Solhigh · Claude Code
88.5
3Gemini 3.7 Flashhigh · Wiz Cyber Model Arena ADK React; simple ReAct loop built with ADK; single bash tool; no cyber-specific tools; general system prompt
85.1
4GPT-5.6 Solhigh · Wiz Cyber Model Arena ADK React; simple ReAct loop built with ADK; single bash tool; no cyber-specific tools; general system prompt
85.1
5Claude Opus 5high · Claude Code
81.6
6Gemini 3.5 Flashhigh · Claude Code
81.6
7Gemini 3.6 Flashhigh · Wiz Cyber Model Arena ADK React; simple ReAct loop built with ADK; single bash tool; no cyber-specific tools; general system prompt
81.6
8GPT-6 Lunahigh · Codex
80.5
9GPT-5.6 Terrahigh · Claude Code
79.3
10GPT-6.1 Solhigh · Claude Code
79.3
11Opus 4.8high · Claude Code
74.7
12Gemini 4 Argonhigh · Wiz Cyber Model Arena ADK React; simple ReAct loop built with ADK; single bash tool; no cyber-specific tools; general system prompt
74.7
13GPT-6 Astrahigh · Codex
71.3
14Sonnet 5high · Claude Code
67.8
15Grok 4.6high · Wiz Cyber Model Arena ADK React; simple ReAct loop built with ADK; single bash tool; no cyber-specific tools; general system prompt
67.8
16GLM-5.3high · Claude Code
65.5
17DeepSeek V4 Pro 0813high · Claude Code
62.1
18DeepSeek V4.1 Flash 1017high · Wiz Cyber Model Arena ADK React; simple ReAct loop built with ADK; single bash tool; no cyber-specific tools; general system prompt
60.9
19Kimi K3high · Claude Code
60.9
20Claude Haiku 5.5high · Claude Code
52.9
21Sonnet 4.5high · Claude Code
50.6
22GLM-5.3 Flashhigh · Claude Code
49.4
23DeepSeek V4.1 Flashhigh · Claude Code
44.8
24MiMo-V2.6-Prohigh · Wiz Cyber Model Arena ADK React; simple ReAct loop built with ADK; single bash tool; no cyber-specific tools; general system prompt
31.0
Loading Atlas data…