Ramp Accounting Bench Pass@1
Finance & Accounting
Share of Ramp Accounting Bench's 411 episodes (137 tasks, three attempts each) that meet every rubric criterion. Kept separate from the partial-credit Mean@3.
Top models (higher is better)
| Model | Score |
|---|---|
| Gemini 4 Argon | 23.4 |
| Claude Fable 5.1 | 12.4 |
| Qwen3.8-Max | 12.4 |
| DeepSeek V4.1 Flash | 12.4 |
| GLM-5.3 | 11.7 |
| GPT-6 Astra | 11.7 |
| Claude Opus 5.5 | 10.9 |
| GPT-6.1 Sol | 10.9 |
| Gemini 3.8 Flash | 10.9 |
| Kimi K3 | 10.2 |
| Gemini 3.7 Flash | 9.5 |
| Grok 4.7 | 9.5 |
| Claude Sonnet 5.5 | 8.0 |
| GLM-5.3 Flash | 7.3 |
| Claude Haiku 5.5 | 7.3 |
| GPT-6 Luna | 7.3 |