Code Llama Python 13B
Meta · 2023-08-24 · 13B parameters
Code Llama - Python 13B is the Python-specialized 13-billion-parameter checkpoint of Meta's Code Llama family, released on August 24, 2023. It starts from the Code Llama 13B code-training stage, trains on a further 100B tokens of Python-heavy data, then goes through the same 16,384-token long-context stage; it does not support infilling.
Benchmark scores
| Benchmark | Score |
|---|---|
| Capability | 97.1 |
| GSM8K | 22.1 |
| HumanEval | 43.3 |
| HumanEval (pass@10) | 77.4 |
| HumanEval (pass@100) | 94.1 |
| MBPP | 49.0 |
| MBPP (pass@10) | 74.0 |
| MBPP (pass@100) | 87.6 |
| MultiPL-E HumanEval (Bash) | 13.9 |
| MultiPL-E HumanEval (C#) | 29.8 |
| MultiPL-E HumanEval (C++) | 39.1 |
| MultiPL-E HumanEval (Java) | 37.3 |
| MultiPL-E HumanEval (PHP) | 33.5 |
| MultiPL-E HumanEval (TypeScript) | 35.2 |