Cerebras GPT
7 models2023
About
The Cerebras GPT family includes seven open-source large language models, ranging from 111 million to 13 billion parameters. These models were developed by Cerebras Systems using the Chinchilla formula, optimizing 20 tokens per parameter to achieve high accuracy within a defined compute budget. Available on Hugging Face under the Apache 2.0 license, these models are accessible for both research and commercial use. Training took place on the Andromeda AI supercomputer, leveraging Cerebras' weight streaming technology for efficient computation across multiple nodes. This setup enhances training speed, reduces costs, and minimizes energy consumption, making them notably efficient compared to other models available 12.
Specifications(7 models)
| Model | Released | Parameters | Reasoning | Code Exec |
|---|---|---|---|---|
| Cerebras GPT 13B | 2023-03 | 13B | No | No |
| Cerebras GPT 7B | 2023-03 | 7B | No | No |
| Cerebras GPT 2.7B | 2023-03 | 2.7B | No | No |
| Cerebras GPT 1.3B | 2023-03 | 1.3B | No | No |
| Cerebras GPT 590M | 2023-03 | — | Yes | Yes |
| Cerebras GPT 256M | 2023-03 | — | No | No |
| Cerebras GPT 111M | 2023-03 | — | No | No |
Frequently Asked Questions
- What is Cerebras GPT?
- The Cerebras GPT family includes seven open-source large language models, ranging from 111 million to 13 billion parameters. These models were developed by Cerebras Systems using the Chinchilla formula, optimizing 20 tokens per parameter to achieve high accuracy within a defined compute budget. Available on Hugging Face under the Apache 2.0 license, these models are accessible for both research and commercial use. Training took place on the Andromeda AI supercomputer, leveraging Cerebras' weight streaming technology for efficient computation across multiple nodes. This setup enhances training speed, reduces costs, and minimizes energy consumption, making them notably efficient compared to other models available 12.
- How many models are in the Cerebras GPT family?
- The Cerebras GPT family contains 7 models.
- What is the latest Cerebras GPT model?
- The latest model is Cerebras GPT 13B, released in 2023-03.
