A-Fantasia (Valid Attempts with Retries)
Games · 2026-09-30
A-Fantasia mean of chess, cube and backward-spelling error rates under the retry/valid-attempt protocol. Each task samples 100 items, allows five additional attempts for unscorable responses and divides by valid attempts only. Selects the latest run with at least 80 valid items on every task. Structured results retain full precision and bootstrap confidence intervals; variable denominators and missing attempt logs prevent an exact standard error.
Top models (lower is better)
| Model | Score |
|---|---|
| Claude Opus 5 | 29.7 |
| Opus 4.6 | 29.7 |
| Opus 4.8 | 34.7 |
| GPT-5.5 | 34.7 |
| Opus 4.1 | 35.3 |
| Opus 4.5 | 35.3 |
| GPT-5.6 Sol | 36.0 |
| GPT-4.5 | 37.7 |
| Opus 4 | 40.0 |
| Sonnet 4.5 | 40.3 |
| GPT-5.4 | 42.3 |
| GPT-5.6 Terra | 43.3 |
| Sonnet 4 | 44.0 |
| Claude 3.7 Sonnet | 45.3 |
| Claude 3.5 Sonnet (Oct 2024) | 46.0 |
| Qwen3.7-Max (source unspecified) | 46.0 |
| Grok 3 (API) | 47.1 |
| Sonnet 4.6 | 47.7 |
| GPT-4o (source unspecified) | 47.9 |
| Opus 3 | 50.3 |
| GPT-5.1 | 51.0 |
| GPT-5.2 | 51.3 |
| GPT-5 Chat | 51.7 |
| Gemini 2.0 Flash | 52.3 |
| GPT-4.1 | 53.0 |
| Gemini 2.5 Flash | 55.3 |
| Kimi K2.6 | 56.0 |
| GPT-5.6 Luna | 56.3 |
| DeepSeek-V4-Pro | 56.7 |
| Kimi K2 Instruct | 57.0 |
| Haiku 4.5 | 58.0 |
| Qwen3.6 Plus (2026-04-02) | 58.0 |
| Hy4 Preview | 58.7 |
| DeepSeek V4.1 Flash | 59.3 |
| Qwen3.7-Plus (source unspecified) | 61.7 |
| Gemini 1.5 Pro (source unspecified) | 62.3 |
| Inkling | 64.7 |
| Gemini 2.0 Flash-Lite | 65.0 |
| Mistral Large 4 Preview | 65.7 |
| GLM-5.1 | 65.9 |