Chartography
Multimodal · 2026-07-16
Professional graphical-reasoning benchmark testing whether models can interpret domain-native charts such as Kaplan-Meier curves, candlestick charts, contour maps, Sankey diagrams, Bode plots, growth curves, and wind roses. It reports mean pass@1 over repeated trials of the 100 distinct tasks. The launch paper uses 20 trials and the current official README describes 10, so the exact repeat count must be established for each source observation.
Top models (higher is better)
| Model | Score |
|---|---|
| Kimi K3.1 Pro | 90.2 |
| Claude Opus 5.5 | 89.0 |
| Claude Fable 5 | 85.2 |
| Claude Opus 5 | 83.0 |
| DeepSeek V4.1 Flash 1017 | 83.0 |
| DeepSeek V4.1 Flash | 78.9 |
| Opus 4.8 | 75.0 |
| Gemini 4 Carbon | 72.5 |
| Gemini 4 Argon | 71.6 |
| GPT-6 Astra | 71.0 |
| Claude Sonnet 5.5 | 61.6 |
| GPT-6 Sol | 53.6 |
| Claude Haiku 5.5 | 46.4 |
| Claude Fable 5.1 | 46.2 |
| GPT-5.6 Sol | 45.0 |
| Gemini 3.8 Flash | 40.9 |
| Gemini 3.7 Flash | 40.4 |
| Gemini 3.6 Flash | 34.0 |
| GPT-5.6 Terra | 34.0 |
| Muse Spark 1.2 | 32.1 |
| GPT-5.6 Luna | 31.4 |
| GPT-5.5 | 31.0 |
| GPT-5.4 | 29.6 |
| GPT-6 Luna | 29.1 |
| Muse Spark 1.3 | 27.6 |
| Kimi K3 | 26.6 |
| Gemini 3.1 Pro Preview | 26.1 |
| Muse Spark 1.1 | 24.4 |
| Grok 4.6 | 20.5 |
| Muse Glimmer 30B | 17.2 |
| Grok 4.5 | 17.0 |
| Sonnet 5 | 16.6 |
| Opus 4.7 | 16.5 |
| GLM-5.3 Flash | 16.3 |
| Qwen3.5 Plus (2026-02-15) | 15.9 |
| Qwen3.7-Plus (source unspecified) | 15.8 |
| Grok 4.7 | 14.7 |
| Grok 4.3 | 12.9 |
| Kimi K2.5 | 12.6 |
| Kimi K2.6 | 12.2 |