Cerebras GPT 13B
Cerebras GPT 13B is released 2023-03-13 in the Cerebras GPT family with open-source; evaluate it while provider pricing coverage matures.
- Family
- Cerebras GPT
- Released
- 2023-03-13
- Context
- 2K
- Parameters
- 13B
- Architecture
- Decoder Only
- Knowledge cutoff
- 2020
- Specialization
- general
- Openness
- Open source
- License
- Apache 2.0OSI-approvedCommercial use: permitted
- Weights
- Unknown
- Code
- Unknown
- Training
- Fine-tuned
No tracked provider pricing is available yet.
About
The Cerebras GPT 13B is a transformer-based large language model developed by Cerebras Systems, notable for its full attention mechanism across all layers, diverging from the sparse banded attention of GPT-3 models. Part of a series ranging from 111 million to 13 billion parameters, it utilizes a GPT-3 style architecture and byte pair encoding for tokenization, following Chinchilla scaling laws for compute optimality. The model was trained with Cerebras' weight streaming technology, enabling efficient data parallelism across nodes.
Providers
No active provider routes are tracked for this model yet.
Capabilities
No model capability flags are currently sourced.
Benchmark peer barsfor Coding
No task-mapped benchmark peers are available for this model yet.
Migration checks
No linked migration route is available for this model yet.