DeepSeekMath 7B Base
DeepSeek · 2024-02-05 · 6.9B parameters
DeepSeekMath 7B Base is a math-specialized 7B model released February 5, 2024, initialized from DeepSeek-Coder-Base-v1.5 7B and further pretrained on 500B tokens, 120B of them from the DeepSeekMath Corpus mined from Common Crawl. It keeps the DeepSeek LLM 7B dense architecture (6,910,365,696 parameters, 30 layers, 4,096-token context).
Benchmark scores
| Benchmark | Score |
|---|---|
| BBH (BIG-Bench Hard) | 59.5 |
| Capability | 99.0 |
| CMATH | 71.7 |
| GSM8K | 64.2 |
| HumanEval | 40.9 |
| MATH | 36.2 |
| MBPP | 52.6 |
| MMLU | 54.9 |