RecurrentGemma
About
RecurrentGemma is a family of open-weight language models developed by Google DeepMind, known for their cutting-edge Griffin architecture. This hybrid design blends linear recurrences with local attention mechanisms, allowing the models to excel in a range of language tasks with reduced memory overhead and efficient inference, especially on lengthy sequences. Unlike traditional transformer models that require memory scaling linearly with sequence length, RecurrentGemma maintains a fixed-sized state, resulting in faster processing speeds. Both pre-trained and instruction-tuned variants are available, the latter being tailored for tasks like dialogue and instruction following. Accessible through platforms like Hugging Face and Kaggle, RecurrentGemma-2B achieves performance akin to Gemma-2B despite being trained on fewer tokens, demonstrating its efficiency and versatility 23910.
Specifications(2 models)
| Model | Released | Parameters |
|---|---|---|
| RecurrentGemma 9B | 2024-06 | 9B |
Available From(1 provider)
Frequently Asked Questions
- What is RecurrentGemma?
- RecurrentGemma is a family of open-weight language models developed by Google DeepMind, known for their cutting-edge Griffin architecture. This hybrid design blends linear recurrences with local attention mechanisms, allowing the models to excel in a range of language tasks with reduced memory overhead and efficient inference, especially on lengthy sequences. Unlike traditional transformer models that require memory scaling linearly with sequence length, RecurrentGemma maintains a fixed-sized state, resulting in faster processing speeds. Both pre-trained and instruction-tuned variants are available, the latter being tailored for tasks like dialogue and instruction following. Accessible through platforms like Hugging Face and Kaggle, RecurrentGemma-2B achieves performance akin to Gemma-2B despite being trained on fewer tokens, demonstrating its efficiency and versatility 23910.
- How many models are in the RecurrentGemma family?
- The RecurrentGemma family contains 2 models.
- What is the latest RecurrentGemma model?
- The latest model is RecurrentGemma 9B, released in 2024-06.
