Qwen2.5 3B
Alibaba · 2024-09-19 · 3.1B parameters
Qwen2.5-3B is a base (pretrained) checkpoint of Alibaba's Qwen2.5 series, released on September 19, 2024 under the Qwen Research license. It is a dense decoder-only Transformer with 3,085,938,688 parameters, grouped-query attention, tied embeddings and a 32,768-token context length.
Benchmark scores
| Benchmark | Score |
|---|---|
| ARC-Challenge | 56.5 |
| BBH (BIG-Bench Hard) | 56.3 |
| Capability | 100.3 |
| GPQA - release-reported, subset unspecified | 26.3 |
| GSM8K | 79.1 |
| HellaSwag (Unspecified Scoring Protocol) | 74.6 |
| HumanEval | 42.1 |
| HumanEval+ | 36.0 |
| MATH | 42.6 |
| MBPP | 57.1 |
| MBPP+ | 49.4 |
| MMLU | 65.6 |
| MMLU Pro | 34.6 |
| MMLU STEM | 62.5 |
| MMLU-Redux | 63.7 |
| MultiPL-E (HumanEval, 8-language average) | 41.2 |
| TheoremQA | 27.4 |
| TruthfulQA (Unspecified Metric) | 48.9 |
| WinoGrande | 71.1 |