Qwen2.5-Coder-32B
Alibaba · 2024-11-12 · 32.8B parameters
Qwen2.5-Coder-32B is Alibaba's 32.8-billion-parameter base code model, released with the expanded Qwen2.5-Coder family on November 12, 2024. Qwen reports that the family was continued-pretrained from Qwen2.5 on over 5.5 trillion tokens of code, text-code grounding, synthetic and math data; the released configuration is set to 32,768 tokens and supports extrapolation to 131,072 tokens with YaRN.
Benchmark scores
| Benchmark | Score |
|---|---|
| ARC-Challenge | 70.5 |
| BigCodeBench Complete | 53.6 |
| BigCodeBench-Hard Complete | 26.4 |
| Capability | 116.2 |
| GSM8K | 91.1 |
| HellaSwag (Unspecified Scoring Protocol) | 83.0 |
| HumanEval | 65.9 |
| HumanEval+ | 60.4 |
| MATH | 57.2 |
| MBPP+ | 68.2 |
| MMLU | 79.1 |
| MMLU Pro | 50.4 |
| MMLU STEM | 75.1 |
| MMLU-Redux | 77.5 |
| MultiPL-E (HumanEval, 8-language average) | 63.9 |
| TheoremQA | 43.1 |
| TruthfulQA (Unspecified Metric) | 54.2 |
| WinoGrande | 80.8 |