Atlas

Benchmarks

SWE-Together (User Correction)

User Correction on SWE-Together: for each run, the number of the simulated user's messages tagged as corrections plus 0.2 for each implicit nudge, averaged over the two runs per task and then over the 109 tasks; lower means the agent needed less steering. A Gemini 3.1 Pro tagger labels the messages of a Gemini 3.1 Pro user simulator on this board, so the measure depends on those seats as well as the agent. Diagnostic; the primary `swe-together` is pass@1.

Models
18
Top score
1.4
Leader
Grok 4.6
Updated
—
Resolution
—
Saturation
—

No signal or difficulty estimate. Held out of the fit by design.

Organizations
18 dated models · 6 frontier modelsClosed weightsOpen weights

SWE-Together (User Correction) leaderboard

Related benchmarks

Loading Atlas data…