Qwen2-57B-A14B Instruct
Alibaba · 2024-06-07 · 57.4B parameters
Qwen2-57B-A14B-Instruct is the instruction-tuned Mixture-of-Experts model of Alibaba's Qwen2 series, released on June 7, 2024. It has 57,408,658,944 total parameters with about 14 billion active per token (8 of 64 routed experts plus a shared expert), and Qwen documents a 65,536-token context length.
Benchmark scores
| Benchmark | Score |
|---|---|
| AlignBench (v1.0 or version unstated) | 7.4 |
| AlignBench v1.1 | 7.0 |
| Arena-Hard-Auto v0.1 | 17.8 |
| C-Eval | 80.5 |
| Capability | 111.4 |
| GPQA - release-reported, subset unspecified | 34.3 |
| HumanEval | 79.9 |
| IFEval (prompt-level strict accuracy) | 59.9 |
| LiveBench 2024-08-31 | 31.1 |
| LiveCodeBench - May 2023 to September 2024 | 22.5 |
| LiveCodeBench v1 | 25.5 |
| MATH | 49.1 |
| MBPP | 70.9 |
| MMLU | 75.4 |
| MMLU Pro | 52.8 |
| MMLU-Redux | 72.6 |
| MT-Bench | 8.6 |
| MultiPL-E (HumanEval, 8-language average) | 66.4 |
| TheoremQA | 33.1 |