Benchmarks
347 sources · 1,258 models × 3,038 benchmarks · 158,912 scoresMathArena's October 2026 research-mathematics release: 47 questions, run under the August native-harness protocol with Python and SageMath and no internet, a 12-hour and $100 limit per problem, and Gemini 3.8 Flash answer-equivalence grading. Published scores average observed correctness across problem attempts; the count records distinct problems, not a single-run binomial denominator.
160th strongest of 798 fitted benchmarks
Signal-to-noise +11.12 for the ArXivMath — Overall family
Another benchmark represents this family in the fit.
| # | Model | Setting | Score | Source |
|---|---|---|---|---|
| 1 | Claude Fable 5.2 | max · MathArena Claude Code 2.1.321; Docker with Python and SageMath; no internet; OpenRouter restricted to Anthropic; 128000 output-token environment setting | 93.6 | |
| 2 | Gemini 4 Carbon | high · MathArena Antigravity CLI 1.5.5; Docker with Python and SageMath; no internet | 93.6 | |
| 3 | GPT-6.1 Sol | max · MathArena Codex; Docker with Python and SageMath; no internet | 90.4 | |
| 4 | Gemini 4 Argon | high · MathArena Antigravity CLI 1.3.5; Docker with Python and SageMath; no internet | 83.0 | |
| 5 | Claude Opus 5.5 | high · MathArena native CLI evaluation; Docker with Python and SageMath; no internet; exact CLI implementation and version unreported | 79.8 | |
| 6 | Kimi K3.1 Pro | max · MathArena Kimi Code 2.3.1; Docker with Python and SageMath; no internet; OpenRouter pinned to MoonshotAI's mxfp4 endpoint without provider fallback | 45.7 | |
| 7 | DeepSeek V4.1 Flash 1017 | max · DeepSeek Harness | 38.3 | |
| 8 | Inkling 2 Small | 0.99 · MathArena OpenCode 1.19.5; Tinker API; Docker with Python and SageMath; no internet; temperature 1.0 | 27.7 | |
| 9 | Muse Spark Open | x-high · MathArena Muse CLI 1.4.9-R6297.1; Docker with Python and SageMath; no internet; OpenRouter pinned to Together without provider fallback | 19.1 |