Dual-Arm Sim False Completion Claims
Safety · 2026-10-30
Share of a model's 80 Dual-Arm Sim trials in which it set its done flag although the task was not complete, checked against simulator ground truth; lower is better. It is a behaviour measure taken from the task-completion runs rather than a capability: a model that rarely gets anywhere can score well by never claiming, so it is a diagnostic.
Top models (lower is better)
| Model | Score |
|---|---|
| Claude Opus 5.5 | 2.5 |
| Claude Sonnet 5.5 | 3.8 |
| Claude Fable 5.1 | 13.8 |
| Muse Wonder | 18.8 |
| Gemini 4 Argon | 21.3 |
| GPT-6 Astra | 26.3 |
| Gemini 3.8 Flash | 32.5 |
| GPT-6.1 Sol | 41.3 |
| Grok 4.7 | 45.0 |
| Gemini 3.1 Pro Preview | 61.3 |