Qwen2-7B
Alibaba · 2024-06-07 · 7.6B parameters
Qwen2-7B is the 7B-class dense base (pretrained) checkpoint of Alibaba's Qwen2 series, released on June 7, 2024. It has 7,615,616,512 parameters with grouped-query attention, was pretrained with 32,768-token sequences, and Qwen documents extrapolated operation up to 131,072 tokens; Qwen2-7B-Instruct is its instruction-tuned counterpart.
Benchmark scores
| Benchmark | Score |
|---|---|
| ARC-Challenge | 60.6 |
| C-Eval | 83.2 |
| Capability | 105.9 |
| CMMLU | 83.9 |
| HellaSwag (Unspecified Scoring Protocol) | 80.7 |
| HumanEval | 51.2 |
| HumanEval+ | 43.3 |
| MBPP+ | 51.9 |
| MMLU | 70.3 |
| MMLU STEM | 64.2 |
| MMLU-Redux | 68.1 |
| TruthfulQA (Unspecified Metric) | 54.2 |
| WinoGrande | 77.0 |