Atlas

Benchmarks

Co-Design Bench

arXiv

Co-Design Bench (Fudan University, Alibaba Group and collaborators) asks whether a coding agent can turn a real user's multimodal request into a complete, working revision of an existing web application. Its 273 tasks, curated from 26,314 real coding sessions, form two tracks: 100 Multimodal Editing tasks, localized edits with an annotated screenshot and optional image, audio, video or document attachments, and 173 Repository Editing tasks, coordinated changes across multi-file frontend repositories. Each final application is built and rendered, then graded against hidden expert-written acceptance criteria (2,331 in all) from source and browser evidence, and a separate LLM verifier challenges every positive verdict. Strict completion, the headline, is the share of a task's criteria judged PASS, averaged over tasks with equal weight. Scores are for a model in the authors' shared Claude Code harness, not for the model alone.

Models
10
Top score
69.9
Leader
GPT-6 Astra
Updated
Oct 26, 2026
Resolution
~5 mo
Useful
Saturation
2027–2031+
Projected
Signal
30thpercentile
205 benchmarks from 0.8 to 3.6219 benchmarks from 3.6 to 6.4134 benchmarks from 6.4 to 9.393 benchmarks from 9.3 to 12.155 benchmarks from 12.1 to 14.925 benchmarks from 14.9 to 17.713 benchmarks from 17.7 to 20.55 benchmarks from 20.5 to 23.37 benchmarks from 23.3 to 26.27 benchmarks from 26.2 to 29.01 benchmark from 29.0 to 31.83 benchmarks from 31.8 to 34.66 benchmarks from 34.6 to 37.43 benchmarks from 37.4 to 40.21 benchmark from 40.2 to 43.11 benchmark from 45.9 to 48.71 benchmark from 65.6 to 68.4

547th strongest of 779

Signal-to-noise +3.91

Difficulty
41stpercentile
1 benchmark from 5.9 to 14.81 benchmark from 23.6 to 32.52 benchmarks from 32.5 to 41.31 benchmark from 41.3 to 50.22 benchmarks from 50.2 to 59.08 benchmarks from 59.0 to 67.97 benchmarks from 67.9 to 76.819 benchmarks from 76.8 to 85.623 benchmarks from 85.6 to 94.541 benchmarks from 94.5 to 103.331 benchmarks from 103.3 to 112.239 benchmarks from 112.2 to 121.148 benchmarks from 121.1 to 129.973 benchmarks from 129.9 to 138.859 benchmarks from 138.8 to 147.6118 benchmarks from 147.6 to 156.5127 benchmarks from 156.5 to 165.3113 benchmarks from 165.3 to 174.235 benchmarks from 174.2 to 183.117 benchmarks from 183.1 to 191.99 benchmarks from 191.9 to 200.81 benchmark from 200.8 to 209.64 benchmarks from 209.6 to 218.5

457th hardest of 779

Capability needed ≲ 143.7

10 dated models · 5 frontier modelsClosed weightsOpen weights
Data: Co-Design Bench Authorsatlas.village.dev
#ModelSettingScoreSource
1GPT-6 Astranot reported · Claude Code
69.9
Co-Design Bench leaderboard (arXiv 2610.35138v1, Tables 2 and 5)
2Claude Opus 5not reported · Claude Code
66.1
Co-Design Bench leaderboard (arXiv 2610.35138v1, Tables 2 and 5)
3Qwen3.8 Max (0902)not reported · Claude Code
65.3
Co-Design Bench leaderboard (arXiv 2610.35138v1, Tables 2 and 5)
4Qwen3.8 27Bnot reported · Claude Code
64.4
Co-Design Bench leaderboard (arXiv 2610.35138v1, Tables 2 and 5)
5Kimi K3not reported · Claude Code
62.9
Co-Design Bench leaderboard (arXiv 2610.35138v1, Tables 2 and 5)
6GPT-5.6 Solnot reported · Claude Code
61.1
Co-Design Bench leaderboard (arXiv 2610.35138v1, Tables 2 and 5)
7Opus 4.8not reported · Claude Code
58.1
Co-Design Bench leaderboard (arXiv 2610.35138v1, Tables 2 and 5)
8GLM-5.3 Flashnot reported · Claude Code
57.3
Co-Design Bench leaderboard (arXiv 2610.35138v1, Tables 2 and 5)
9GLM-5.2not reported · Claude Code
56.7
Co-Design Bench leaderboard (arXiv 2610.35138v1, Tables 2 and 5)
10Grok 4.6not reported · Claude Code
51.1
Co-Design Bench leaderboard (arXiv 2610.35138v1, Tables 2 and 5)
Loading Atlas data…