Arena-Hard-Auto v2.0 - Hard Prompts
Chat & Writing · 2025-04-23
The 500 hard prompts of Arena-Hard-Auto v2.0. Each answer is compared pairwise with the o3-mini baseline by an LLM judge (GPT-4.1 by default), and the score is the model's estimated win rate against the baseline in percent. A component of the Arena-Hard v2.0 family (arena-hard-v2); v0.1 uses a different prompt set, baseline and judge. Higher is better.
Top models (higher is better)
| Model | Score |
|---|---|
| Nemotron Cascade 2 30B A3B | 88.2 |
| Nemotron 3 Nano 30B A3B | 72.1 |
| gpt-oss-20b | 71.2 |
| Qwen3.5 35B A3B | 64.5 |
| Opus 4 | 59.7 |
| Kimi K2 Instruct | 54.5 |
| GPT-4.1 | 51.7 |
| Sonnet 4 | 51.6 |
| Qwen3 30B A3B Thinking 2507 | 49.6 |
| DeepSeek-V3-0324 | 39.9 |
| Qwen3-235B-A22B | 39.9 |