← All benchmarks

SWE-Clar (implementation)

Code · 2026-10-22

Upwork's SWE-Clar gives a coding agent 100 greenfield software tasks rebuilt from real Upwork jobs that each paid at least US$4,000. Each task starts from an incomplete client request: the agent questions a simulated client, whose replies a supervisor agent reviews, until it is ready, and then builds the software from that frozen clarification context with no further client contact. The implementation score is the share of tasks whose delivered software passes all of the task's hidden acceptance tests.

Top models (higher is better)