ProgramBench (problem counts) - Fully Resolved

Name: ProgramBench (problem counts) - Fully Resolved
Published: 2026-05-05

Code · 2026-05-05

The Fully Resolved split of ProgramBench (problem counts). This child benchmark reports raw counts of tasks whose submitted implementation passes all hidden behavioral tests.

Top models (higher is better)

Model	Score
Claude Sonnet 5.5	13.0
GPT-6 Astra	11.0
GPT-6.1 Sol	6.0
Claude Haiku 5.5	5.0
Gemini 4 Argon	5.0
Muse Spark 1.3	5.0
GPT-6 Sol	4.0
Kimi K3	4.0
GLM-5.3	3.0
GPT-5.6 Sol	3.0
Opus 4.8	2.0
Gemini 3.8 Flash	2.0
Grok 4.6	2.0
Sonnet 4.6	1.0
DeepSeek V4.1 Flash	1.0
GLM-5.2	1.0
GPT-5.4	1.0
GPT-5.5	1.0
GPT-5.6 Terra	1.0
GPT-6 Luna	1.0
Grok 4.7	1.0
Inkling-Small	1.0
MiMo-V2.6-Flash	1.0
MiMo-V2.6-Pro	1.0
Muse Spark 1.2	1.0
Muse Wonder Full	1.0
Haiku 4.5	0.0
Opus 4.7	0.0
Sonnet 5	0.0
DeepSeek-V4-Flash-0731	0.0
DeepSeek V4 Pro 0813	0.0
DeepSeek-V4-Pro	0.0
Gemini 3.1 Flash-Lite Preview	0.0
Gemini 3.1 Pro Preview	0.0
Gemini 3.5 Flash	0.0
Gemini 3.5 Flash-Lite	0.0
Gemini 3.6 Flash	0.0
Gemini 3.7 Flash	0.0
Gemini 3 Flash Preview	0.0
GLM-5.1	0.0