Pythia 160M
Pythia 160M is released 2023-05-31 in the Pythia family with open-source; evaluate it while provider pricing coverage matures.
- Family
- Pythia
- Released
- 2023-05-31
- Context
- 2K
- Parameters
- 160M
- Architecture
- Decoder Only
- Knowledge cutoff
- 2020-03
- Specialization
- general
- Openness
- Open source
- License
- Apache 2.0OSI-approvedPermitted
- Weights
- Unknown
- Code
- Unknown
- Training
- Fine-tuned
No tracked provider pricing is available yet.
About
The Pythia 160M is a large language model by EleutherAI and part of the Pythia Scaling Suite, consisting of models from 70 million to 12 billion parameters. Built on the GPT-NeoX architecture, it features 12 layers, 768 hidden dimensions, and 12 attention heads, supporting a context length of 2048 tokens. It is trained with The Pile dataset, totaling 299,892,736,000 tokens, across 154 checkpoints. While it excels in text generation and interpretability research, its limitations include generating biased or harmful content, being English-only, and not being fine-tuned for deployment in specific applications.
Providers
No active provider routes are tracked for this model yet.
Capabilities
No model capability flags are currently sourced.
Benchmark peer barsfor Coding
No task-mapped benchmark peers are available for this model yet.
Migration checks
No linked migration route is available for this model yet.
No tracked provider pricing is available yet.