RULER - 256K
General QA · 2024-04-09
RULER average accuracy over its 13 synthetic long-context tasks generated at a 256K-token context length. The number of samples per task (RULER's default or a smaller count such as 100) is a run setting recorded in notes. Higher is better.
Top models (higher is better)
| Model | Score |
|---|---|
| Nemotron 3 Nano 30B A3B | 92.9 |
| Nemotron 3 Ultra 550B A55B Base | 86.2 |
| Nemotron 3.5 Lightning 30B A3B Base | 76.9 |