Qwen2.5 0.5B
Alibaba · 2024-09-19 · 494M parameters
Qwen2.5-0.5B is the smallest base (pretrained) checkpoint of Alibaba's Qwen2.5 series, released on September 19, 2024 after pretraining on up to 18 trillion tokens. It is a dense decoder-only Transformer with 494,032,768 parameters, grouped-query attention, tied embeddings and a 32,768-token context length.
Benchmark scores
| Benchmark | Score |
|---|---|
| ARC-Challenge | 35.6 |
| BBH (BIG-Bench Hard) | 20.3 |
| Capability | 77.1 |
| GPQA - release-reported, subset unspecified | 24.8 |
| GSM8K | 41.6 |
| HellaSwag (Unspecified Scoring Protocol) | 52.1 |
| HumanEval | 30.5 |
| HumanEval+ | 26.8 |
| MATH | 19.5 |
| MBPP | 39.3 |
| MBPP+ | 33.8 |
| MMLU | 47.5 |
| MMLU Pro | 15.7 |
| MMLU STEM | 39.8 |
| MMLU-Redux | 45.1 |
| MultiPL-E (HumanEval, 8-language average) | 18.9 |
| TheoremQA | 16.0 |
| TruthfulQA (Unspecified Metric) | 40.2 |
| WinoGrande | 56.3 |