← All models

Nemotron-H 56B Base 8K

NVIDIA · 2025-04-14 · 56.3B parameters

Nemotron-H 56B Base 8K is NVIDIA's open-weight 56B hybrid Mamba-Transformer base model and the largest Nemotron-H checkpoint, released on April 14, 2025; the BF16 checkpoint contains exactly 56,324,350,464 parameters. Its 118 layers are mostly Mamba-2 and MLP layers with ten self-attention layers, and it was the first Nemotron model fully pretrained with an FP8 per-tensor current-scaling recipe, over a 20-trillion-token horizon with an 8,192-token context. The weights are released under the NVIDIA Internal Scientific Research and Development Model License.

Benchmark scores