← All models

GPT-2 (124M)

OpenAI · 2019-02-14 · 124M parameters

GPT-2 (124M) is the smallest variant of OpenAI's GPT-2 language model family, released with sampling code alongside the February 2019 GPT-2 announcement. It is a 12-layer decoder-only transformer trained on the WebText corpus with a next-token prediction objective. The GPT-2 paper lists it as 117M; OpenAI later corrected the count to 124M in the openai/gpt-2 repository.

Benchmark scores