← All models

Moonlight 16B A3B Instruct

Moonshot AI · 2025-02-22 · 16B parameters

Moonlight-16B-A3B-Instruct is the instruction-tuned chat checkpoint of Moonshot AI's Moonlight Mixture-of-Experts model, released in February 2025 alongside the base weights and the Muon is Scalable for LLM Training report. It shares the base model's DeepSeek-V3-style MoE architecture with MLA attention, about 16B total and 3B activated parameters, 5.7-trillion-token Muon pre-training, and 8K context, and was post-trained with Muon. Its BF16 tensors contain 15,960,111,936 parameters.

Benchmark scores