Atlas

Models

← All models

Code Llama Instruct 13B

Meta · 2023-08-24 · 13B parameters

Code Llama - Instruct 13B is the instruction-tuned 13-billion-parameter checkpoint from Meta's August 2023 Code Llama release, intended for natural-language-driven code generation and safer assistant use. It is fine-tuned from the long-context Code Llama 13B on about 5B tokens of instruction, self-instruct coding and safety data, and keeps the base model's infilling ability and 100,000-token inference support.

Benchmark scores

BenchmarkScore
Capability98.1
HumanEval42.7
HumanEval (pass@10)71.6
HumanEval (pass@100)91.6
MBPP49.4
MBPP (pass@10)71.2
MBPP (pass@100)84.1
MultiPL-E HumanEval (Bash)13.9
MultiPL-E HumanEval (C#)24.0
MultiPL-E HumanEval (C++)42.2
MultiPL-E HumanEval (Java)40.5
MultiPL-E HumanEval (PHP)32.3
MultiPL-E HumanEval (TypeScript)39.0
ToxiGen - toxic generation rate0.0
TruthfulQA generation - % informative91.9
TruthfulQA generation - % truthful44.3
TruthfulQA generation - % truthful and informative36.8
Loading Atlas data…