EDIT-Bench Complete Hard

Name: EDIT-Bench Complete Hard
Published: 2025-11-06

Code · 2025-11-06

The EDIT-Bench Complete hard slice reports pass@1 on the 245 tasks classified as hard by the benchmark authors.

Model	Score
Sonnet 4	30.2
Claude 3.5 Sonnet (Oct 2024)	25.7
GPT-5	23.3
Kimi K2 Instruct 0905	22.9
DeepSeek-V3.1	22.0
Gemini 2.5 Pro	22.0
GLM 4.6	22.0
Claude 3.7 Sonnet	21.2
Qwen3-Coder-480B-A35B-Instruct	20.4
Qwen3 Coder Flash (rolling alias)	20.0
GPT-4o (2024-08-06)	19.6
GPT-5 Mini	19.6
o3-mini	19.6
Sonnet 4.5	19.2
Llama 4 Maverick Instruct	15.9
O4 Mini	15.9
Grok Code Fast 1	15.1
Llama 3.1 405B Instruct	15.1
gpt-oss-20b	14.7
Grok 4 Fast	14.7
Mistral Small 3.2 24B Instruct 2506	14.7
DeepSeek-R1-0528	14.3
GPT-5 Nano	14.3
Llama 3.3 70B Instruct	13.9
Codestral 25.08	13.5
gpt-oss-120b	13.5
GPT-4o Mini	13.1
Llama 4 Scout Instruct	12.2
Qwen2.5 72B Instruct	11.4
Qwen2.5 Coder 32B Instruct	11.4
Gemma 3 27B IT	10.6
Devstral Medium 1.0	9.4
GLM-4.5	9.0
Llama 3.1 8B Instruct	9.0
Devstral Small 1.1	8.2
Qwen3-30B-A3B	7.8
Gemma 3n E4B Instruct	6.9
Gemma 3 12B IT	6.5
Qwen3 14B	6.1
Kimi-Dev-72B	5.7