← All models

Nemotron-H 8B Base 8K

NVIDIA · 2025-04-14 · 8.1B parameters

Nemotron-H 8B Base 8K is NVIDIA's open-weight 8B hybrid Mamba-Transformer base model from the Nemotron-H family, released on April 14, 2025; the BF16 checkpoint contains exactly 8,100,852,736 parameters. Its 52 layers are mostly Mamba-2 and MLP layers with four self-attention layers, and NVIDIA trained it on a 15-trillion-token horizon of curated and synthetic data with an 8,192-token context. The weights are released under the NVIDIA Internal Scientific Research and Development Model License.

Benchmark scores