o11y-bench 1.1 Pass^3
Agents
Grafana Labs' open benchmark of 64 observability and SRE tasks across logs, metrics, traces, dashboards and incidents. An agent works against a real Grafana stack, and each task runs three times and is graded by checks plus an LLM judge. Pass^3 is the share of tasks where all three attempts pass, and timeouts, crashes and run errors count as failures. Version 1.1 changed the judge and added a task, so it isn't comparable with 1.0.
Benchmark source: O11ybench