WebArena-Verified
Agents
ServiceNow's re-audited release of the WebArena web-agent benchmark: 812 tasks on the original self-hosted shopping, shopping-admin, Reddit, GitLab, Wikipedia and map sites, with every task, reference answer and evaluator revised. Deterministic evaluators score the agent's response and captured network traces instead of LLM judging or substring matching. The 258-task Hard subset is a different item set.
Top models (higher is better)
| Model | Score |
|---|---|
| Opus 4.8 | 71.2 |
| Muse Spark 1.1 | 69.0 |
| Gemini 3.1 Pro Preview | 69.0 |
| GPT-5.5 | 67.0 |
| Muse Spark | 59.0 |
| LFM3-1.7B | 38.4 |
| Qwen3.6 35B A3B | 37.6 |
| Qwen3.5-9B | 28.9 |
| Nemotron 3.5 Lightning 30B A3B (precision unspecified) | 28.7 |
| LFM2.5-24B-A2B | 26.8 |
| Gemma 4 26B A4B IT | 25.6 |
| Qwen3.5-4B | 21.9 |
| LFM2.5-2.6B | 3.0 |