RULER - 128K
General QA · 2024-04-09
RULER average accuracy over its 13 synthetic retrieval, multi-hop tracing, aggregation and question-answering tasks generated at a 128K-token input length. It is the primary of the RULER family because more models report it than the 4K-128K average; the other lengths and the average are fallback components. The number of samples per task is a run setting recorded in notes. Higher is better.