← All models

Nemotron-H 47B Base 8K

NVIDIA · 2025-04-12 · 46.8B parameters

Nemotron-H 47B Base 8K is NVIDIA's open-weight 47B hybrid Mamba-Transformer base model, released on April 12, 2025; the BF16 checkpoint contains exactly 46,791,554,816 parameters. NVIDIA produced it by pruning Nemotron-H 56B Base with its MiniPuzzle method and distilling it on 63 billion tokens in FP8, leaving 98 layers with five self-attention layers and an 8,192-token context, so that it can run in FP4 on a single 32 GiB GPU. Because its cumulative training exposure combines the parent's pretraining with the distillation run, no training-token total is stored; the weights are released under the NVIDIA Internal Scientific Research and Development Model License.

Benchmark scores