← All models

Code Llama 34B

Meta · 2023-08-24 · 33.7B parameters

Code Llama 34B is the 34-billion-parameter base checkpoint of Meta's Code Llama family, released on August 24, 2023. It continues training the unreleased Llama 2 34B on 500B tokens of code-heavy data without the infilling objective, and a long-context stage at 16,384-token sequences with a RoPE base period of 1,000,000 lets Meta's inference code run it on inputs of up to 100,000 tokens.

Benchmark scores