Benchmarks
331 sources · 1,258 models × 2,980 benchmarks · 155,643 scoresDuelLab's GameBench 2 gives a model the rules of each of eight public strategy games (Go, Chess, Shogi, Xiangqi, Hex, International Draughts, Game of the Amazons and Tumbleweed in suite 2-known8-r3) and the interface its player must use; the model writes a player program through the provider's API with no agent harness, and DuelLab checks, builds and runs it, sending repair requests when it fails. The programs play deterministic head-to-head matches across seats, setups and opponents, and a Bradley-Terry fit of every rated match in a game gives each program a 0-100 score, 100 times its expected points against a fixed reference field of programs. A model-and-setting row's headline score is the equal-weight mean over games, with each generation attempt that failed for a model cause counted as 0 beside the working programs. Scores are relative to the field and refitted at every release, so Atlas imports one immutable release at a time, one row per model and recorded reasoning request.
332nd strongest of 785 fitted benchmarks
Signal-to-noise +7.00
330th hardest of 785 fitted benchmarks
Capability needed 155.2