ARC-AGI-2
General QA · 2025-03-24
A harder successor to ARC-AGI that tests few-shot abstract reasoning and pattern generalization on grid-based tasks, with an added emphasis on efficiency of compute per task solved. The official scorer gives each task the fraction of its test pairs solved by either permitted attempt, then averages these fractional task scores across the 120-task evaluation cohort. The task count does not describe binary successes.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-6 Astra | 95.0 |
| GPT-6.1 Sol | 94.2 |
| Claude Opus 5.5 | 93.3 |
| GPT-5.6 Sol | 92.5 |
| Gemini 4 Argon | 91.3 |
| Claude Opus 5 | 90.4 |
| Claude Fable 5.1 | 90.0 |
| GPT-6 Sol | 89.6 |
| Claude Fable 5 | 89.2 |
| Gemini 3.8 Flash | 89.2 |
| GPT-5.5 | 85.0 |
| Gemini 3.7 Flash | 84.6 |
| GPT-5.5 Pro | 84.2 |
| GPT-5.6 Terra | 83.9 |
| Claude Sonnet 5.5 | 83.9 |
| DeepSeek V4.1 Flash 1017 | 78.1 |
| Gemini 3.1 Pro Preview | 77.1 |
| dots3-note Preview | 76.8 |
| DeepSeek V4.1 Flash | 72.9 |
| Gemini 3.5 Flash | 72.1 |
| Opus 4.8 | 72.1 |
| Opus 4.6 | 69.2 |
| Opus 4.7 | 68.3 |
| GPT-5.4 | 67.5 |
| Grok 4.6 | 67.1 |
| GLM-5.3 Flash | 65.8 |
| Grok 4.20 | 65.1 |
| DeepSeek-V4-Flash-0731 | 61.4 |
| Grok 4.7 | 61.4 |
| Claude Haiku 5.5 | 61.3 |
| DeepSeek V4 Pro 0813 | 61.3 |
| Sonnet 4.6 | 60.4 |
| Gemini 3.6 Flash | 60.4 |
| Kimi K3 | 60.4 |
| GPT-5.6 Luna | 59.5 |
| GPT-6 Luna | 59.3 |
| Muse Spark Open | 57.5 |
| Grok 4.5 | 52.6 |
| GPT-5.2 | 43.3 |
| Qwen3.8 27B | 42.4 |