Code Llama Python 7B
Meta · 2023-08-24 · 6.7B parameters
Code Llama - Python 7B is the Python-specialized 7-billion-parameter checkpoint of Meta's Code Llama family, released on August 24, 2023. It starts from the Code Llama 7B code-training stage, trains on a further 100B tokens of Python-heavy data, then goes through the same 16,384-token long-context stage; it does not support infilling.
Benchmark scores
| Benchmark | Score |
|---|---|
| Capability | 94.8 |
| GSM8K | 13.0 |
| HumanEval | 38.4 |
| HumanEval (pass@10) | 70.3 |
| HumanEval (pass@100) | 90.6 |
| MBPP | 47.6 |
| MBPP (pass@10) | 70.3 |
| MBPP (pass@100) | 84.8 |
| MultiPL-E HumanEval (Bash) | 16.5 |
| MultiPL-E HumanEval (C#) | 24.7 |
| MultiPL-E HumanEval (C++) | 32.3 |
| MultiPL-E HumanEval (Java) | 35.4 |
| MultiPL-E HumanEval (PHP) | 32.3 |
| MultiPL-E HumanEval (TypeScript) | 23.9 |