Pythia Models by EleutherAI
Last refreshed 2026-05-19. Next refresh: weekly.
Details
About
The Pythia large language model (LLM) family, crafted by EleutherAI, comprises 16 models tailored for in-depth research into the nuances of LLM behavior and training dynamics. The models range from 70 million to 12 billion parameters, all trained on the Pile dataset, with the inclusion and exclusion of deduplication, ensuring a uniform data sequence. This consistency allows for comprehensive studies on how scaling parameters affect model performance in a meticulously controlled setting. While not designed for optimal downstream tasks, the Pythia models offer performance akin to other equivalent-sized LLMs and serve primarily educational and research purposes. Publicly accessible, they provide extensive checkpoints and insights into the training process, though they remain not fine-tuned for specific applications and largely cater to English language processing.
Decision facts
- Best fit
- codingchatbot and role-playing use cases
- Capability starting point
- Pythia 1.4B with 2K context
- Lowest tracked input
- Pythia 12B · $0.200/1M · Fireworks AI
- Closest related family
- GPT-Neo
Current Variants
Use-when guidance is based on each model's tracked capabilities, context window, release date, and replacement status.
Use when the workload needs 2K context and 1.4B parameters.
Use when the workload needs 2K context and 12B parameters.
Use when the workload needs 2K context and 14M parameters.
Use when the workload needs 2K context and 160M parameters.
Use when the workload needs 2K context and 1B parameters.
Use when the workload needs 2K context and 2.8B parameters.
Use when the workload needs 2K context and 31M parameters.
Use when the workload needs 2K context and 410M parameters.
Use when the workload needs 2K context and 6.9B parameters.
Use when the workload needs 2K context and 70M parameters.
| Model | Use when | Released | Signals | Status |
|---|---|---|---|---|
| Pythia 1.4B | Use when the workload needs 2K context and 1.4B parameters. | 2023-05 | 2K context1.4B parameters | Current |
| Pythia 12B | Use when the workload needs 2K context and 12B parameters. | 2023-05 | 2K context12B parameters | Current |
| Pythia 14M | Use when the workload needs 2K context and 14M parameters. | 2023-05 | 2K context14M parameters | Current |
| Pythia 160M | Use when the workload needs 2K context and 160M parameters. | 2023-05 | 2K context160M parameters | Current |
| Pythia 1B | Use when the workload needs 2K context and 1B parameters. | 2023-05 | 2K context1B parameters | Current |
| Pythia 2.8B | Use when the workload needs 2K context and 2.8B parameters. | 2023-05 | 2K context2.8B parameters | Current |
| Pythia 31M | Use when the workload needs 2K context and 31M parameters. | 2023-05 | 2K context31M parameters | Current |
| Pythia 410M | Use when the workload needs 2K context and 410M parameters. | 2023-05 | 2K context410M parameters | Current |
| Pythia 6.9B | Use when the workload needs 2K context and 6.9B parameters. | 2023-05 | 2K context6.9B parameters | Current |
| Pythia 70M | Use when the workload needs 2K context and 70M parameters. | 2023-05 | 2K context70M parameters | Current |
Release Timeline
1 release groupSpecifications(10 models)
| Model | Released | Context | Parameters |
|---|---|---|---|
| Pythia 1.4B | 2023-05 | 2K | 1.4B |
| Pythia 12B | 2023-05 | 2K | 12B |
| Pythia 14M | 2023-05 | 2K | 14M |
| Pythia 160M | 2023-05 | 2K | 160M |
| Pythia 1B | 2023-05 | 2K | 1B |
| Pythia 2.8B | 2023-05 | 2K | 2.8B |
| Pythia 31M | 2023-05 | 2K | 31M |
| Pythia 410M | 2023-05 | 2K | 410M |
| Pythia 6.9B | 2023-05 | 2K | 6.9B |
| Pythia 70M | 2023-05 | 2K | 70M |
Available From(1 provider)
Pricing
| Model | Provider | Input / 1M | Output / 1M | Type |
|---|---|---|---|---|
| Pythia 12B | Fireworks AI | $0.2 | $0.2 | Provisioned |
Models(10)
Pythia 1.4B
Pythia 12B
Pythia 14M
Pythia 160M
Pythia 1B
Pythia 2.8B
Pythia 31M
Pythia 410M
Pythia 6.9B
Pythia 70M