Ruby LLM Program Fixer — School Library
Code · 2025-08-01
The School Library Program Fixer task asks a model to repair a broken Ruby school-library implementation against 28 tests. Its composite is 90% test success plus 10% RuboCop quality, with each offense subtracting two quality points down to zero.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Opus 5 | 83.5 |
| GPT-5.5 | 81.1 |
| Claude Fable 5 | 80.1 |
| Opus 4.7 | 80.1 |
| Claude Sonnet 5.5 | 80.1 |
| DeepSeek-V3.2-Speciale | 80.1 |
| Claude 3 Haiku | 79.9 |
| Claude Opus 5.5 | 79.9 |
| DeepSeek-V4-Pro | 79.7 |
| GLM-5 | 79.7 |
| GPT-5.6 Luna | 79.7 |
| GPT-5.6 Terra | 79.7 |
| Kimi K3 | 79.7 |
| Grok 4 | 79.5 |
| Kimi K2.6 | 79.5 |
| GLM-5.3 Flash | 79.3 |
| GPT-5.6 Luna Pro | 79.1 |
| GPT-5.6 Terra Pro | 79.1 |
| Sonnet 4.6 | 78.9 |
| DeepSeek-V4-Flash | 78.5 |
| Horizon Beta | 78.1 |
| Claude Fable 5.1 | 77.9 |
| Opus 4.6 | 76.9 |
| Muse Spark 1.3 | 76.7 |
| GPT-5.2 Instant (source unspecified) | 76.7 |
| GPT-6 Sol | 76.7 |
| Opus 4.8 | 76.5 |
| GLM-5.1 | 76.5 |
| Qwen3.8-Max | 76.5 |
| Gemma 4 31B IT | 76.3 |
| GLM-5.2 | 76.3 |
| GPT-5.1 Instant | 76.3 |
| GPT-5.3 Instant (source unspecified) | 76.3 |
| Qwen3.6 Plus (2026-04-02) | 76.3 |
| Sonnet 4.5 | 76.1 |
| DeepSeek-R1 | 76.1 |
| gpt-oss-120b | 76.1 |
| O4 Mini | 76.1 |
| o1-mini | 75.9 |
| Grok 3 Mini | 75.7 |