Code Llama 7B
Meta · 2023-08-24 · 6.7B parameters
Code Llama 7B is the smallest base checkpoint of Meta's Code Llama family, released on August 24, 2023. It continues training Llama 2 7B on 500B tokens of code-heavy data with a fill-in-the-middle objective, so it supports both left-to-right completion and infilling, and a long-context stage at 16,384-token sequences with a RoPE base period of 1,000,000 lets Meta's inference code run it on inputs of up to 100,000 tokens.