Cerebras GPT 111M
Cerebras GPT 111M is released 2023-03-13 in the Cerebras GPT family with open-source; evaluate it while provider pricing coverage matures.
- Family
- Cerebras GPT
- Released
- 2023-03-13
- Context
- 2K
- Parameters
- 111M
- Architecture
- Decoder Only
- Knowledge cutoff
- 2020
- Specialization
- general
- Openness
- Open source
- License
- Apache 2.0OSI-approvedPermitted
- Weights
- Unknown
- Code
- Unknown
- Training
- Fine-tuned
No tracked provider pricing is available yet.
About
The Cerebras GPT 111M is a transformer-based AI model from Cerebras Systems, noted for its compact architecture and efficient performance. It features 10 layers and utilizes a GPT-3 style framework with 768 hidden units and 12 attention heads, able to process sequences up to 2048 tokens. Trained on "The Pile" dataset, it adheres to the Chinchilla scaling laws for computational efficiency, offering robust text generation and few-shot learning capabilities. Its open-source nature, released under the Apache 2.0 license, promotes research and development use, with weight streaming technology facilitating training across multiple nodes for scalability.
Providers
No active provider routes are tracked for this model yet.
Capabilities
No model capability flags are currently sourced.
Benchmark peer barsfor Coding
No task-mapped benchmark peers are available for this model yet.
Migration checks
No linked migration route is available for this model yet.