MacroscopeBench Recall
Code · 2026-09-22
MacroscopeBench known-bug recall: the share of the dataset's known bugs that an LLM judge matches to one of the reviewer's findings, each bug credited at most once per run, averaged over three runs, in percent. Every model faces the same known bugs, so it is a component of the MacroscopeBench F1, but it does not penalise invalid findings. The number of known bugs behind the public table is not stated.
Top models (higher is better)
| Model | Score |
|---|---|
| Claude Opus 5.5 | 83.8 |
| DeepSeek V4.1 Flash | 83.1 |
| Claude Opus 5 | 76.6 |
| GLM-5.3 | 76.4 |
| GPT-6.1 Terra | 75.0 |
| GPT-6 Sol | 73.8 |
| GPT-5.6 Sol | 70.8 |
| Grok 4.6 | 69.9 |
| GPT-6.1 Sol | 69.2 |
| GPT-6 Astra | 67.8 |
| Kimi K3 | 66.7 |
| Muse Wonder | 64.8 |
| Gemini 3.8 Flash | 64.1 |
| GPT-6 Luna | 61.1 |