OpenAI-MRCR 2-needle - 128K
General QA · 2025-04-14
OpenAI-MRCR (multi-round co-reference resolution) inserts two identical needle requests into a long synthetic conversation and asks the model to reproduce the response to a specified needle. This row is the 128K bin, averaging the string match ratio over prompts with 64K-128K input tokens. Higher is better.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-5 | 95.2 |
| GPT-5 Mini | 84.3 |
| GPT-4.1 | 57.2 |
| O4 Mini | 56.4 |
| o3 | 55.0 |
| GPT-4.1 Mini | 47.2 |
| GPT-5 Nano | 43.2 |
| GPT-4.5 | 38.5 |
| GPT-4.1 Nano | 36.6 |
| GPT-4o (2024-11-20) | 31.9 |
| GPT-4o Mini | 24.5 |
| o1 (2024-12-17) | 22.1 |
| o3-mini | 18.7 |