ForecastBench Preliminary Dataset Brier Index
Forecasting · 2024-09-30
Difficulty-adjusted Brier Index on resolved dataset questions from preliminary tournament submissions. These are early results for the same submitted forecasts later represented in tournament results, with potentially different resolution windows. Retained as a diagnostic rather than a second independent capability factor.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-6 Astra | 63.9 |
| GPT-6 Sol | 63.8 |
| GPT-5.5 | 63.4 |
| Grok 4.20 | 63.0 |
| Muse Spark 1.3 | 62.9 |
| Claude Opus 5.5 | 62.5 |
| Gemini 3.7 Flash | 62.0 |
| Opus 4.8 | 61.7 |
| Claude Opus 5 | 61.7 |
| Gemini 3.6 Flash | 61.7 |
| GPT-5.6 Sol | 61.7 |
| Grok 4.6 | 61.5 |
| Gemini 3.1 Pro Preview | 61.3 |
| Grok 4.5 | 61.3 |
| Gemini 2.5 Pro Preview 03-25 | 61.2 |
| GPT-4.5 | 61.1 |
| GPT-5 | 61.0 |
| Opus 4 | 60.8 |
| GPT-5.6 Sol Pro | 60.7 |
| Grok 4.3 | 60.7 |
| Kimi K3 | 60.7 |
| Gemini 2.5 Flash Preview 04-17 | 60.6 |
| Gemini 3.5 Flash | 60.6 |
| Gemini 3 Pro Preview | 60.6 |
| O4 Mini | 60.6 |
| GLM-5 | 60.4 |
| GPT-6 Luna | 60.4 |
| o3 | 60.3 |
| Sonnet 4.6 | 60.2 |
| Gemini 3 Flash Preview | 60.2 |
| GPT-4.1 | 60.2 |
| Grok 4.1 Fast | 60.2 |
| Opus 4.1 | 60.1 |
| Grok 4 Fast | 60.1 |
| Claude 3.7 Sonnet | 60.1 |
| Opus 4.7 | 60.0 |
| GPT-5 Mini | 60.0 |
| Grok 4 | 60.0 |
| MiniMax M3 | 60.0 |
| Opus 4.5 | 59.9 |