Qwen3 4B Base
Alibaba · 2025-04-29 · 4B parameters
Qwen3-4B-Base is the pre-trained (not instruction-tuned) 4B dense checkpoint of Alibaba's Qwen3 family, released April 29, 2025 under Apache 2.0 alongside the post-trained Qwen3-4B. Qwen describes a three-stage pre-training on about 36 trillion tokens in 119 languages, ending with long-context training at 32,768 tokens, which is the checkpoint's configured context length.
Benchmark scores
| Benchmark | Score |
|---|---|
| BBH (BIG-Bench Hard) | 72.6 |
| Capability | 112.3 |
| GSM8K | 87.8 |
| MATH | 54.1 |
| MGSM | 67.7 |
| MMLU | 73.0 |
| MMLU Pro | 50.6 |
| MMLU-Redux | 72.8 |
| SuperGPQA | 28.4 |