RULER - 512K
General QA · 2024-04-09
RULER average accuracy over its 13 synthetic long-context tasks generated at a 512K-token context length. The number of samples per task (RULER's default or a smaller count such as 100) is a run setting recorded in notes. Higher is better.
Top models (higher is better)
| Model | Score |
|---|---|
| Nemotron 3 Nano 30B A3B | 91.3 |
| Nemotron 3 Ultra 550B A55B Base | 84.5 |
| Nemotron 3 Nano 30B A3B Base | 70.6 |