Benchmarks
347 sources · 1,258 models × 3,038 benchmarks · 158,912 scoresSWE-Together (Meta) rebuilds 109 repository-level coding tasks in 27 repositories from real multi-turn sessions between users and coding agents. The agent starts from the session's first user message, and a user simulator (Gemini 3.1 Pro on this board) replays the original user's follow-ups, clarifications and corrections while the agent works in a sandbox. An agentic Claude Opus 4.6 judge scores each final patch against a frozen, weighted task rubric, and a run solves its task at a judge score of 0.85 or above. The score is pass@1, the share of tasks solved averaged over two runs per task, with every model in the opencode harness. Scores are comparable only within one user simulator.
584th strongest of 798 fitted benchmarks
Signal-to-noise +3.69
534th hardest of 798 fitted benchmarks
Capability needed ≲ 134.6
| # | Model | Setting | Score | Source |
|---|---|---|---|---|
| 1 | Claude Fable 5.1 | high · OpenCode | 69.3 | |
| 2 | Claude Fable 5 | high · OpenCode | 68.8 | |
| 3 | Claude Opus 5.5 | high · OpenCode | 68.8 | |
| 4 | Grok 4.7 | high · OpenCode | 65.1 | |
| 5 | Gemini 3.8 Flash | high · OpenCode | 64.2 | |
| 6 | Claude Opus 5 | high · OpenCode | 63.8 | |
| 7 | Kimi K3 | high · OpenCode | 63.8 | |
| 8 | Opus 4.6 | high · OpenCode | 62.4 | |
| 9 | Claude Sonnet 5.5 | high · OpenCode | 62.4 | |
| 10 | Muse Spark 1.3 | high · OpenCode | 62.4 | |
| 11 | Opus 4.7 | high · OpenCode | 61.5 | |
| 12 | Opus 4.8 | high · OpenCode | 61.5 | |
| 13 | GPT-6 Astra | high · OpenCode | 60.6 | |
| 14 | GPT-6 Sol | high · OpenCode | 60.6 | |
| 15 | Grok 4.6 | high · OpenCode | 60.6 | |
| 16 | GPT-6 Luna | high · OpenCode | 59.6 | |
| 17 | GPT-6.1 Sol | high · OpenCode | 58.3 | |
| 18 | GPT-5.6 Sol | high · OpenCode | 57.8 |
Benchmarks outside its family whose model rankings track it most reliably (Spearman ρ over at least 10 shared models).