Needle-in-a-Haystack multi-needle (Meta protocol)
General QA · 2024-07-23
Meta's multi-needle variant of Needle-in-a-Haystack used for Llama 3.1 and later: four needles are inserted into a long document and the model must retrieve two of them, at context lengths evenly spaced from 2,000 to 131,072 tokens (128,000 for non-Llama models) in ten steps. Scores are average recall across the lengths. It is a synthetic retrieval probe close to saturation and is kept as a diagnostic.