DeepSeek LLM 7B Base
DeepSeek · 2023-11-29 · 6.9B parameters
DeepSeek LLM 7B Base is the 7B dense base model of DeepSeek's first general LLM release, made public on November 29, 2023. It is a 30-layer LLaMA-style decoder with multi-head attention and a 4,096-token context, pretrained from scratch on a reported 2 trillion mainly English and Chinese tokens; computed from its configuration it has 6,910,365,696 parameters.
Benchmark scores
| Benchmark | Score |
|---|---|
| AGIEval | 26.4 |
| ARC-Challenge | 48.1 |
| ARC-Easy | 67.9 |
| BBH (BIG-Bench Hard) | 39.5 |
| C-Eval | 45.0 |
| C3 | 65.4 |
| Capability | 81.6 |
| CCPM | 76.9 |
| ChID | 89.3 |
| CLUEWSC | 73.1 |
| CMATH | 34.5 |
| CMMLU | 47.2 |
| DROP | 41.0 |
| GSM8K | 17.4 |
| HellaSwag (Unspecified Scoring Protocol) | 75.4 |
| HumanEval | 26.2 |
| MBPP | 39.0 |
| MMLU | 48.2 |
| Natural Questions (open-domain, closed-book) | 22.2 |
| OpenBookQA | 55.8 |
| PIQA | 79.2 |
| RACE-High | 46.5 |
| RACE-Middle | 63.2 |
| TriviaQA | 59.7 |
| WinoGrande | 70.5 |