Atlas

Benchmarks

OpenWorldEngineer — Software

Mercor

OpenWorldEngineer Mean@8 on its 8 software tasks (data engineering, databases, languages and systems: streaming ETL, a Parquet lakehouse, a join engine, a SPARQL store, SQL change replay, a bytecode VM, function layout and an embedded graphics renderer).

Models
15
Top score
62.2
Leader
Claude Fable 5.2
Updated
—
Resolution
—
Saturation
—
Signal
95thpercentile
192 benchmarks from 0.8 to 3.5218 benchmarks from 3.5 to 6.1128 benchmarks from 6.1 to 8.895 benchmarks from 8.8 to 11.567 benchmarks from 11.5 to 14.128 benchmarks from 14.1 to 16.814 benchmarks from 16.8 to 19.49 benchmarks from 19.4 to 22.111 benchmarks from 22.1 to 24.72 benchmarks from 24.7 to 27.46 benchmarks from 27.4 to 30.13 benchmarks from 30.1 to 32.73 benchmarks from 32.7 to 35.45 benchmarks from 35.4 to 38.01 benchmark from 38.0 to 40.71 benchmark from 40.7 to 43.41 benchmark from 43.4 to 46.01 benchmark from 62.0 to 64.6

42nd strongest of 785

Signal-to-noise +19.67 for the OpenWorldEngineer (Mean@8) family

Difficulty
Not estimated

Another benchmark represents this family in the fit.

Organizations
15 dated models · 6 frontier modelsClosed weightsOpen weights
Data: Mercoratlas.village.dev
#ModelSettingScoreSource
1Claude Fable 5.2max · OpenWorld harness (OpenWorldEngineer runner) on Harbor 0.23 with Modal sandboxes; web search over an October 1, 2026 web snapshot as the only network access; task time budgets of 4 to 18 hours
62.2
OpenWorldEngineer official leaderboard
2Claude Opus 5.5max · OpenWorld harness (OpenWorldEngineer runner) on Harbor 0.23 with Modal sandboxes; web search over an October 1, 2026 web snapshot as the only network access; task time budgets of 4 to 18 hours
49.7
OpenWorldEngineer official leaderboard
3GPT-6 Astramax · OpenWorld harness (OpenWorldEngineer runner) on Harbor 0.23 with Modal sandboxes; web search over an October 1, 2026 web snapshot as the only network access; task time budgets of 4 to 18 hours
44.3
OpenWorldEngineer official leaderboard
4GPT-6.1 Solmax · OpenWorld harness (OpenWorldEngineer runner) on Harbor 0.23 with Modal sandboxes; web search over an October 1, 2026 web snapshot as the only network access; task time budgets of 4 to 18 hours
43.1
OpenWorldEngineer official leaderboard
5Claude Sonnet 5.5max · OpenWorld harness (OpenWorldEngineer runner) on Harbor 0.23 with Modal sandboxes; web search over an October 1, 2026 web snapshot as the only network access; task time budgets of 4 to 18 hours
42.2
OpenWorldEngineer official leaderboard
6Claude Fable 5.1max · OpenWorld harness (OpenWorldEngineer runner) on Harbor 0.23 with Modal sandboxes; web search over an October 1, 2026 web snapshot as the only network access; task time budgets of 4 to 18 hours
36.6
OpenWorldEngineer official leaderboard
7Gemini 4 Carbonhigh · OpenWorld harness (OpenWorldEngineer runner) on Harbor 0.23 with Modal sandboxes; web search over an October 1, 2026 web snapshot as the only network access; task time budgets of 4 to 18 hours
31.5
OpenWorldEngineer official leaderboard
8Gemini 4 Argonhigh · OpenWorld harness (OpenWorldEngineer runner) on Harbor 0.23 with Modal sandboxes; web search over an October 1, 2026 web snapshot as the only network access; task time budgets of 4 to 18 hours
22.1
OpenWorldEngineer official leaderboard
9Kimi K3.1 Promax · OpenWorld harness (OpenWorldEngineer runner) on Harbor 0.23 with Modal sandboxes; web search over an October 1, 2026 web snapshot as the only network access; task time budgets of 4 to 18 hours
16.1
OpenWorldEngineer official leaderboard
10DeepSeek V4.1 Flashmax · OpenWorld harness (OpenWorldEngineer runner) on Harbor 0.23 with Modal sandboxes; web search over an October 1, 2026 web snapshot as the only network access; task time budgets of 4 to 18 hours
14.7
OpenWorldEngineer official leaderboard
11GLM-5.3max · OpenWorld harness (OpenWorldEngineer runner) on Harbor 0.23 with Modal sandboxes; web search over an October 1, 2026 web snapshot as the only network access; task time budgets of 4 to 18 hours
14.6
OpenWorldEngineer official leaderboard
12Muse Wondermax · OpenWorld harness (OpenWorldEngineer runner) on Harbor 0.23 with Modal sandboxes; web search over an October 1, 2026 web snapshot as the only network access; task time budgets of 4 to 18 hours
14.2
OpenWorldEngineer official leaderboard
13Grok 4.7x-high · OpenWorld harness (OpenWorldEngineer runner) on Harbor 0.23 with Modal sandboxes; web search over an October 1, 2026 web snapshot as the only network access; task time budgets of 4 to 18 hours
8.4
OpenWorldEngineer official leaderboard
14Beam 1.0 501B A23Bmax · OpenWorld harness (OpenWorldEngineer runner) on Harbor 0.23 with Modal sandboxes; web search over an October 1, 2026 web snapshot as the only network access; task time budgets of 4 to 18 hours
3.6
OpenWorldEngineer official leaderboard
15Inklingnot reported · OpenWorld harness (OpenWorldEngineer runner) on Harbor 0.23 with Modal sandboxes; web search over an October 1, 2026 web snapshot as the only network access; task time budgets of 4 to 18 hours
1.4
OpenWorldEngineer official leaderboard
Loading Atlas data…