Concept Synth INDUCTION Challenge100 Holdout
General QA
Post-selection diagnostic for direct Round-1 Challenge100 formulas against the original (v1) holdout sidecar: train-correct responses verified correct on all available generated holdout worlds, divided by all 100 tasks. The v1 generator left eight tasks without holdout worlds and six with two to four. Tasks without verified holdout success receive no credit, including unavailable holdout worlds; missing outcomes remain unknown in the underlying records. Holdout results are never used for prompting or response selection. Superseded on the live leaderboard by the holdouts-v2 diagnostic.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-6 Astra | 79.0 |
| GPT-6.1 Sol | 62.0 |
| Claude Opus 5.5 | 50.0 |
| Gemini 4 Argon | 37.0 |
| Claude Sonnet 5.5 | 37.0 |
| GPT-6 Sol | 35.0 |
| Claude Fable 5.1 | 30.0 |
| GPT-5.6 Sol | 22.0 |
| Claude Opus 5 | 20.0 |
| Claude Fable 5 | 19.0 |
| GPT-5.6 Terra | 17.0 |
| Grok 4.6 | 17.0 |
| Grok 4.7 | 14.0 |
| Muse Spark 1.1 | 14.0 |
| GPT-6 Luna | 12.0 |
| Gemini 3.7 Flash | 9.0 |
| Grok 4.5 | 8.0 |
| Gemini 3.8 Flash | 8.0 |
| GPT-5.6 Luna | 7.0 |
| Muse Spark 1.2 | 7.0 |
| Gemini 3.5 Flash | 6.0 |
| DeepSeek V4 Pro 0813 | 5.0 |
| DeepSeek-V4-Pro | 4.0 |
| DeepSeek V4.1 Flash | 3.0 |
| Gemini 3.6 Flash | 3.0 |