Qwen2-0.5B
Alibaba · 2024-06-07 · 494M parameters
Qwen2-0.5B is the smallest base (pretrained) checkpoint of Alibaba's Qwen2 series, released on June 7, 2024. It is a dense decoder-only Transformer with 494,032,768 parameters, grouped-query attention and tied embeddings, and Qwen documents a 32,768-token context length; Qwen2-0.5B-Instruct is its instruction-tuned counterpart.
Benchmark scores
| Benchmark | Score |
|---|---|
| C-Eval | 58.2 |
| Capability | 73.1 |
| CMMLU | 55.1 |
| GPQA - release-reported, subset unspecified | 29.8 |
| HumanEval+ | 18.9 |
| MBPP+ | 27.6 |
| MMLU Pro | 14.7 |
| MMLU STEM | 27.5 |
| MMLU-Redux | 40.7 |
| MultiPL-E (HumanEval, 8-language average) | 16.3 |
| TruthfulQA (Unspecified Metric) | 39.7 |