OpenAI-MRCR 2-needle - 1M
General QA · 2025-04-14
OpenAI-MRCR (multi-round co-reference resolution) inserts two identical needle requests into a long synthetic conversation and asks the model to reproduce the response to a specified needle. This row is the 1M bin, the dataset's largest, averaging the string match ratio over prompts with 512K-1M input tokens, as OpenAI reports it under "OpenAI-MRCR: 2 needle 1M". Higher is better.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-4.1 | 46.3 |
| GPT-4.1 Mini | 33.3 |
| GPT-4.1 Nano | 12.0 |