Qwen2.5 Coder 3B Instruct
Alibaba · 2024-11-12 · 3.1B parameters
Qwen2.5-Coder-3B-Instruct is the instruction-tuned 3.1-billion-parameter model of Alibaba's Qwen2.5-Coder family, released on November 12, 2024 under the Qwen Research license. It uses grouped-query attention and tied embeddings with a 32,768-token context.
Benchmark scores
| Benchmark | Score |
|---|---|
| AIME 2024 | 6.7 |
| BigCodeBench Instruct | 35.8 |
| BigCodeBench-Hard Instruct | 14.2 |
| C-Eval | 53.9 |
| Capability | 106.2 |
| GPQA - release-reported, subset unspecified | 28.3 |
| GSM8K | 80.7 |
| HumanEval | 84.1 |
| HumanEval+ | 80.5 |
| IFEval | 44.2 |
| LiveCodeBench - July to November 2024 | 10.8 |
| LiveCodeBench - July to September 2024 | 10.8 |
| MATH | 58.1 |
| MBPP | 73.6 |
| MBPP+ | 62.4 |
| MMLU | 56.5 |
| MMLU Pro | 35.2 |
| MultiPL-E (HumanEval, 8-language average) | 72.1 |