← All models

Nemotron-H 4B Base 8K

NVIDIA · 2025-10-23 · 4.5B parameters

Nemotron-H 4B Base 8K is NVIDIA's open-weight 4B hybrid Mamba-Transformer base model, pruned from Nemotron-H 8B Base with group-aware SSM pruning and distilled on 380 billion tokens; the BF16 checkpoint contains exactly 4,489,223,040 parameters. Its 52 layers are mostly Mamba-2 and MLP layers with four self-attention layers and an 8,192-token context. The card dates its public Hugging Face release to October 23, 2025, and the weights are released under the NVIDIA Internal Scientific Research and Development Model License.

Benchmark scores