CRED v0.92 - Paper–code
Science · 2026-07-20
The CRED v0.92 paper–code inconsistency chapter measures detection recall on errors that become visible only by comparing claims in an empirical policy paper with its analysis code.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-6 Astra | 100.0 |
| GPT-5.6 Sol | 98.9 |
| GPT-5.6 Luna | 96.7 |
| GPT-5.6 Terra | 95.6 |
| Kimi K3 | 95.6 |
| GPT-5.5 | 94.4 |
| Claude Fable 5 | 92.2 |
| GPT-5.6 Luna Pro | 86.7 |
| GLM-5.3 Flash | 81.1 |
| GPT-6 Luna | 80.0 |
| GLM-5.2 | 80.0 |
| Kimi K2.6 | 75.6 |
| Opus 4.8 | 71.1 |
| DeepSeek-V4-Pro | 71.1 |
| DeepSeek-V4-Flash | 67.8 |
| Sonnet 4.6 | 64.4 |
| Gemini 3.8 Flash | 61.1 |
| Nemotron 3 Ultra 550B A55B | 56.7 |
| Sonnet 5 | 55.6 |
| Opus 4 | 55.6 |
| Opus 4.1 | 55.6 |
| Sonnet 4.5 | 55.6 |
| MiniMax M3 | 54.4 |
| Opus 4.5 | 50.0 |
| GLM-5 | 46.7 |
| Gemini 3.7 Flash | 45.6 |
| Kimi K2 Thinking | 44.4 |
| Kimi K2.5 | 44.4 |
| Kimi K2 Instruct | 41.1 |
| Ring 2.6 1T | 41.1 |
| Gemini 3.5 Flash | 36.7 |
| GLM-4.7 | 36.7 |
| DeepSeek-R1-0528 | 32.2 |
| Qwen3.5 397B A17B | 31.1 |
| Hy3 preview | 26.7 |
| GLM-4.5 | 25.6 |
| GLM 4.6 | 25.6 |
| Gemini 3 Flash Preview | 23.3 |
| DeepSeek-V3.2 | 22.2 |
| gpt-oss-120b | 22.2 |