← All benchmarks

EnterpriseOps-Gym

Agents · 2026-03-13

ServiceNow Research's EnterpriseOps-Gym evaluates stateful planning and tool use in a containerized enterprise sandbox with 164 database tables and 512 tools across eight domains (calendar, customer service, drive, email, HR, IT service management, Teams and hybrid workflows). Executable SQL verifiers score the final environment state; the released dataset offers an oracle tool set plus 5-, 10- and 15-distractor-tool modes, and the paper, dataset card and oracle config give different task counts. Distinct from Artificial Analysis' EnterpriseOps-Gym-AA reimplementation in its Stirrup harness.

Top models (higher is better)