KellyBench (2023/24 season, mean ROI)
Forecasting · 2026-04-09
General Reasoning's KellyBench test scenario: an agent bets through the 2023/24 English Premier League season (120 matchdays) in a sandbox, building prediction models from historical match and player data and staking from its bankroll on home, draw, away and over/under 2.5 goals at real bookmaker closing odds, at least one bet per matchday. The score is the mean return on the starting bankroll over five seeds, in percent; stakes come from the current bankroll, so -100 (ruin) is the floor and the return is otherwise unbounded. Rewards are deterministic changes in log-wealth with no LLM grader.
Top models (higher is better)
| Model | Score |
|---|---|
| Opus 4.7 | -3.7 |
| GPT-5.4 | -7.9 |
| GPT-5.5 | -9.6 |
| Opus 4.6 | -11.2 |
| GLM-5.2 | -29.6 |
| DeepSeek-V4-Pro | -46.4 |
| GLM-5 | -51.6 |
| Kimi K2.6 | -60.4 |
| Gemini 3.1 Pro Preview | -66.0 |
| Mistral Medium 3.5 | -77.9 |
| Kimi K2.5 | -89.6 |
| Mistral Large 3 675B Instruct 2512 | -98.8 |