Seal-0
Search · 2025-06-01
Seal-0 is the hardest 111-question subset of SealQA, fact-seeking questions on which web search returns conflicting, noisy or unhelpful results, chosen so that frontier models with search initially answered them all wrongly. Agents answer with web search and the answers are graded for correctness. Scores report accuracy; tool setup and the number of averaged runs are recorded per row. Higher is better.
Top models (higher is better)
| Model | Score |
|---|---|
| Kimi K2.5 | 57.4 |
| Kimi K2 Thinking | 56.3 |
| Opus 4.5 | 47.7 |
| Qwen3.5-27B | 47.2 |
| Qwen3 Max Thinking (2026-01-23) | 46.9 |
| Qwen3.5 397B A17B | 46.9 |
| Gemini 3 Pro Preview | 45.5 |
| gpt-oss-120b | 45.1 |
| GPT-5.2 | 45.0 |
| Qwen3.5 122B A10B | 44.1 |
| Qwen3.5 35B A3B | 41.4 |
| GPT-5 Mini | 34.2 |
| Kimi K2 Instruct 0905 | 25.2 |