← All models

RecurrentGemma 2B

Google · 2024-04-09 · 2.7B parameters

RecurrentGemma 2B is Google DeepMind's open-weight, English text-to-text pre-trained model built on the Griffin architecture, released on April 9, 2024. Instead of global attention it mixes gated linear recurrences with 2,048-token local attention, so its inference state stays fixed in size; it has 2.68 billion parameters (2.03 billion non-embedding), was trained on 2 trillion tokens of 8,192-token sequences, and can sample sequences of arbitrary length.

Benchmark scores