xbench-ScienceQA
Science · 2025-05-25
xbench-ScienceQA is xbench's 100-question PhD-level science benchmark (77 open-ended and 23 multiple-choice questions) written by doctoral students and industry experts, with questions kept out of search indexes and answers graded by an LLM judge. Each model answers every question five times with search disabled, and the headline score is the mean accuracy over the five runs. It is unrelated to the older multimodal ScienceQA benchmark.
Top models (higher is better)
| Model | Score |
|---|---|
| Opus 4.8 | 76.4 |
| Claude Opus 5.5 | 76.4 |
| GPT-6 Astra Pro | 74.6 |
| Claude Fable 5 | 73.4 |
| Qwen3.8-Max | 73.4 |
| Opus 4.7 | 73.0 |
| GPT-5.5 Pro | 73.0 |
| DeepSeek V4.1 Flash | 72.6 |
| Gemini 3.8 Flash | 72.4 |
| Grok 4.5 | 72.4 |
| GPT-5.4 Pro | 72.2 |
| Gemini 3 Pro Preview | 71.6 |
| Kimi K3 | 71.2 |
| DeepSeek-V4-Flash-0731 | 71.0 |
| Doubao Seed 2.1 Pro | 71.0 |
| GPT-5.6 Sol Pro | 71.0 |
| GPT-5.4 | 70.8 |
| Qwen3.7-Max (source unspecified) | 70.8 |
| Grok 4.6 | 70.4 |
| Gemini 3.7 Flash | 70.2 |
| Hy4 Preview | 69.8 |
| Doubao Seed 2.0 Pro | 69.2 |
| Hy3 | 66.6 |
| GLM-5.3 | 65.8 |
| Kimi K2.6 | 65.8 |
| Gemini 3.6 Flash | 65.6 |
| Grok 4 | 65.0 |
| GLM-5.2 | 64.6 |