← All models

Code Llama 70B

Meta · 2024-01-29 · 69B parameters

Code Llama 70B is the largest base checkpoint of Meta's Code Llama family, released on January 29, 2024. It continues training Llama 2 70B on 1T tokens of code-heavy data with a fill-in-the-middle objective, and a long-context stage with a RoPE base period of 1,000,000 lets it run on inputs of up to 100,000 tokens, unlike the 4,096-token Python and Instruct 70B variants.

Benchmark scores