← All models

OLMoE 1B-7B 0125 Instruct

Allen Institute for AI (Ai2) · 6.9B parameters

OLMoE-1B-7B-0125-Instruct is Ai2's January 2025 post-trained checkpoint of its fully open OLMoE mixture-of-experts model, which routes each token to 8 of 64 experts in each of its 16 layers for about 1B active of 7B total parameters. It was built from the OLMoE-1B-7B-0125 base, which adds OLMo 2's Dolmino mix for mid-training, through supervised fine-tuning on an OLMo variant of the Tülu 3 mixture, DPO, and RLVR on GSM8K math data. Ai2 introduced it as the model behind its on-device OLMoE iOS app in February 2025 and released it under Apache 2.0 with a 4,096-token context.

Benchmark scores