← All benchmarks

BabaIsBench Stage 0: The Intro

Games · 2026-07-16

Quesma's BabaIsBench ports the puzzle game Baba Is You to the Harbor framework: an agent reads the board as JSON, plays through a simulator command line that runs the game's own engine, and submits a move string that the verifier replays through that engine; the instructions never name the game or the level. Stage 0, The Intro, is the game's eight opening levels. Every model gets three attempts per level in the Terminus 2 harness via OpenRouter, with a 60-minute limit per attempt, and the score is the share of its 24 attempts that are solved (pass@1). The stage is near ceiling: eight of the 32 models on the September 5, 2026 board solve every attempt.

Top models (higher is better)