Chinchilla
About
The Chinchilla family of large language models, developed by Google DeepMind, was introduced in March 2022. These models are notable for their exploration of the scaling laws in LLM training. Uniquely, they highlighted that for optimal model performance, the size of the model and the number of training tokens should be proportionately scaled. For instance, the Chinchilla model with 70 billion parameters used the same computational resources as a 280 billion parameter Gopher model but was trained on quadruple the data, leading to enhanced performance across numerous benchmarks. This approach challenged the previous assumption that increasing model size inherently improves performance, emphasizing the critical role of ample data in achieving state-of-the-art results 1)23.
Specifications(2 models)
| Model | Released | Parameters |
|---|---|---|
| Gopher 280B | 2022-03 | 280B |
| Chinchilla 70B | 2022-03 | 70B |
Frequently Asked Questions
- What is Chinchilla?
- The Chinchilla family of large language models, developed by Google DeepMind, was introduced in March 2022. These models are notable for their exploration of the scaling laws in LLM training. Uniquely, they highlighted that for optimal model performance, the size of the model and the number of training tokens should be proportionately scaled. For instance, the Chinchilla model with 70 billion parameters used the same computational resources as a 280 billion parameter Gopher model but was trained on quadruple the data, leading to enhanced performance across numerous benchmarks. This approach challenged the previous assumption that increasing model size inherently improves performance, emphasizing the critical role of ample data in achieving state-of-the-art results 1)23.
- How many models are in the Chinchilla family?
- The Chinchilla family contains 2 models.
- What is the latest Chinchilla model?
- The latest model is Gopher 280B, released in 2022-03.
