Pythia
About
The Pythia large language model (LLM) family, crafted by EleutherAI, comprises 16 models tailored for in-depth research into the nuances of LLM behavior and training dynamics. The models range from 70 million to 12 billion parameters, all trained on the Pile dataset, with the inclusion and exclusion of deduplication, ensuring a uniform data sequence. This consistency allows for comprehensive studies on how scaling parameters affect model performance in a meticulously controlled setting. While not designed for optimal downstream tasks, the Pythia models offer performance akin to other equivalent-sized LLMs and serve primarily educational and research purposes. Publicly accessible, they provide extensive checkpoints and insights into the training process, though they remain not fine-tuned for specific applications and largely cater to English language processing.
Specifications(10 models)
| Model | Released | Parameters |
|---|---|---|
| Pythia 12B | 2023-05 | 12B |
| Pythia 6.9B | 2023-05 | 6.9B |
| Pythia 2.8B | 2023-05 | 2.8B |
| Pythia 1.4B | 2023-05 | 1.4B |
| Pythia 1B | 2023-05 | 1B |
| Pythia 410M | 2023-05 | 410M |
| Pythia 160M | 2023-05 | 160M |
| Pythia 70M | 2023-05 | 70M |
| Pythia 31M | 2023-05 | 31M |
| Pythia 14M | 2023-05 | 14M |
Available From(1 provider)
Pricing
| Model | Provider | Input / 1M | Output / 1M | Type |
|---|---|---|---|---|
| Pythia 12B | Fireworks AI | $0.2 | $0.2 | Provisioned |
Frequently Asked Questions
- What is Pythia?
- The Pythia large language model (LLM) family, crafted by EleutherAI, comprises 16 models tailored for in-depth research into the nuances of LLM behavior and training dynamics. The models range from 70 million to 12 billion parameters, all trained on the Pile dataset, with the inclusion and exclusion of deduplication, ensuring a uniform data sequence. This consistency allows for comprehensive studies on how scaling parameters affect model performance in a meticulously controlled setting. While not designed for optimal downstream tasks, the Pythia models offer performance akin to other equivalent-sized LLMs and serve primarily educational and research purposes. Publicly accessible, they provide extensive checkpoints and insights into the training process, though they remain not fine-tuned for specific applications and largely cater to English language processing.
- How many models are in the Pythia family?
- The Pythia family contains 10 models.
- What is the latest Pythia model?
- The latest model is Pythia 12B, released in 2023-05.
- How much does Pythia cost?
- Pythia models are available at $0.2/1M input tokens through providers like Fireworks AI.
