WildBench v2 (HELM Capabilities)
Chat & Writing · 2025-03-20
HELM Capabilities' WildBench run: 1,000 tasks from WildBench v2's real user conversations, each response scored 1 to 10 against WildBench's v2 checklist template by an ensemble of GPT-4o (2024-05-13) and Llama 3.1 405B Instruct Turbo, joined by Claude 3.5 Sonnet (2024-10-22) for runs first published through release v1.14.0; Anthropic deprecated Claude 3.5 Sonnet before v1.15.0. The judges' mean is rescaled from 1-10 to 0-100. The judge ensemble is HELM's own, so these scores are not comparable with WildBench's own leaderboard.
Top models (higher is better)
| Model | Score |
|---|---|
| Qwen3 235B A22B Instruct 2507 | 86.6 |
| GPT-5.1 | 86.3 |
| Kimi K2 Instruct | 86.2 |
| o3 | 86.1 |
| Gemini 3 Pro Preview | 85.9 |
| GPT-5 | 85.7 |
| Gemini 2.5 Pro Preview 03-25 | 85.7 |
| GPT-5 Mini | 85.5 |
| Sonnet 4.5 | 85.4 |
| o4-mini | 85.4 |
| Opus 4 | 85.2 |
| gpt-oss-120b | 84.5 |
| Haiku 4.5 | 83.9 |
| Sonnet 4 | 83.8 |
| DeepSeek-V3 | 83.1 |
| GPT-4o (2024-11-20) | 82.8 |
| DeepSeek-R1-0528 | 82.8 |
| Qwen3-235B-A22B | 82.8 |
| Gemini 2.5 Flash-Lite | 81.8 |
| Gemini 2.5 Flash Preview 04-17 | 81.7 |
| Claude 3.7 Sonnet | 81.4 |
| Gemini 1.5 Pro 002 | 81.3 |
| Qwen3-Next-80B-A3B-Thinking | 80.7 |
| GPT-5 Nano | 80.6 |
| Qwen2.5 72B Instruct | 80.2 |
| Mistral Large 2.1 (Instruct 2411) | 80.1 |
| Gemini 2.0 Flash | 80.0 |
| Llama 4 Maverick Instruct FP8 | 80.0 |
| Claude 3.5 Sonnet (Oct 2024) | 79.2 |
| Gemini 1.5 Flash 002 | 79.2 |
| GPT-4o Mini | 79.1 |
| Gemini 2.0 Flash-Lite Preview 02-05 | 79.0 |
| GLM 4.5 Air FP8 | 78.9 |
| Mistral Small 3.1 24B Instruct 2503 | 78.8 |
| Nova Premier | 78.8 |
| Llama 3.1 405B Instruct | 78.3 |
| Palmyra X5 | 78.0 |
| Llama 4 Scout Instruct | 77.9 |
| Nova Pro | 77.7 |
| Haiku 3.5 | 76.0 |