MRCR v2 (8-needle) - 128K average
General QA · 2025-06-27
MRCR v2 is Google DeepMind’s synthetic multi-round coreference-resolution evaluation for long-context reasoning beyond retrieval. This 8-needle metric is the cumulative average through 128K tokens, using string similarity between the requested prior response and the model’s reproduction.
Top models (higher is better)
| Model | Score |
|---|---|
| Gemini 3.7 Flash | 97.0 |
| GPT-5.5 | 94.8 |
| Fugu Ultra v1.0 | 93.6 |
| Gemini 3.6 Flash | 91.8 |
| Gemini 3.1 Pro Preview | 84.9 |
| Sonnet 4.6 | 84.9 |
| Opus 4.6 | 84.0 |
| Grok 4.5 | 81.4 |
| Gemini 3.5 Flash | 77.3 |
| Gemini 3 Pro Preview | 77.0 |
| GPT-5.6 Luna | 74.8 |
| Gemini 3.5 Flash-Lite | 72.2 |
| Sonnet 5 | 71.6 |
| Gemini 3 Flash Preview | 67.2 |
| GPT-5.1 | 61.6 |
| Gemini 3.1 Flash-Lite Preview | 60.1 |
| Gemini 3.1 Flash-Lite | 60.1 |
| Opus 4.7 | 59.3 |
| Gemini 2.5 Pro | 58.0 |
| o3 | 57.1 |
| Grok 4.1 Fast | 54.6 |
| Gemini 2.5 Flash | 54.3 |
| GPT-5 Mini | 52.5 |
| Gemini 2.5 Flash Preview (09-2025) | 52.4 |
| Sonnet 4.5 | 47.1 |
| GPT-5.4 Mini | 42.7 |
| Sonnet 4 | 39.1 |
| O4 Mini | 36.3 |
| Haiku 4.5 | 35.3 |
| Grok 3 Think (source unspecified) | 34.0 |
| Gemini 2.5 Flash-Lite | 30.6 |
| Gemini 2.5 Flash-Lite Preview (09-2025) | 25.6 |
| Gemini 2.0 Flash | 19.0 |
| Opus 4 | 16.1 |