Qwen1.5 0.5B
Alibaba · 2024-02-04 · 620M parameters
Qwen1.5-0.5B is the smallest base (pretrained) checkpoint of Alibaba's Qwen1.5 family, announced on February 4, 2024. It is a dense decoder-only Transformer with 619,570,176 parameters and a 32,768-token context window, intended for fine-tuning; Qwen1.5-0.5B-Chat is its aligned counterpart.
Benchmark scores
| Benchmark | Score |
|---|---|
| BBH (BIG-Bench Hard) | 18.3 |
| C-Eval | 50.5 |
| Capability | 65.1 |
| CMMLU | 46.6 |
| GSM8K | 22.0 |
| HumanEval | 12.2 |
| MATH | 3.1 |
| MBPP | 6.8 |
| MMLU | 39.2 |