Atlas

Benchmarks

Synthetic Sciences FrontierML (all 25 tasks)

Synthetic Sciences' FrontierML gives an agent 25 open-ended empirical ML research problems built around active research directions, such as credit assignment under sparse terminal rewards, pretraining a language model without backpropagation or unlearning that survives fine-tuning. Each run is 12 hours in Lab0, a virtual lab with H100s rented per job from a shared cluster, a $50-125 wallet, a frozen paper library, a pinned dataset mirror and model APIs, but no validation score, grader or internet; the sealed model, method or recipe it hands in is graded on protected data or at a scale the agent could not afford. Each task's metric is normalized between a starting reference (0) and a strong-result reference set by judgment (1) and clipped to 0-1, and the score is the mean over all 25 tasks of one run each; 15 tasks are released and 10 stay private. Every model runs in Synthetic Sciences' Horizon harness at its highest reasoning effort. Unrelated to Induction Labs' FrontierML (frontierml).

Models
4
Top score
0.5
Leader
Claude Fable 5.2
Updated
Nov 18, 2026
Resolution
—
Saturation
—

No signal or difficulty estimate. Too few models have been measured on this benchmark to estimate its difficulty.

Organizations
4 dated models · 3 frontier modelsClosed weightsOpen weights
Data: Synthetic Sciencesatlas.village.dev

Synthetic Sciences FrontierML (all 25 tasks) leaderboard

#ModelSettingScoreSource
1Claude Fable 5.2max · Horizon 0.4.17 (Synthetic Sciences' FrontierML build of mini-swe-agent, under Harbor)
0.5
2Claude Opus 5.5max · Horizon 0.4.17 (Synthetic Sciences' FrontierML build of mini-swe-agent, under Harbor)
0.4
3GPT-6 Astramax · Horizon 0.4.17 (Synthetic Sciences' FrontierML build of mini-swe-agent, under Harbor)
0.4
4Gemini 4 Carbonhigh · Horizon 0.4.17 (Synthetic Sciences' FrontierML build of mini-swe-agent, under Harbor)
0.4
Loading Atlas data…