MPT Models by Databricks Mosaic
Last refreshed 2026-05-19. Next refresh: weekly.
Details
Links
WebsiteAbout
The MosaicML Pretrained Transformer (MPT) family is a collection of advanced, open-source large language models designed for diverse applications, available for commercial use. These models stand out for their decoder-only architecture reminiscent of GPT models, offering enhanced performance through optimized layer implementations and increased training stability. Notably, the MPT models eliminate context length limitations via ALiBi (Attention with Linear Biases), replacing traditional positional embeddings. The MPT family encompasses the base model MPT-7B, alongside specialized variants like MPT-7B-Instruct, MPT-7B-Chat, and MPT-7B-StoryWriter-65k+, each fine-tuned for distinct tasks ranging from instruction-following to storytelling. They were developed using a vast dataset comprising 1 trillion tokens of text and code, underscoring their capability to process and generate high-quality text outputs 125.
Decision facts
Current Variants
Use-when guidance is based on each model's tracked capabilities, context window, release date, and replacement status.
Use when the workload needs 8k context and 30B parameters.
Use when the workload needs 2k context and 7B parameters.
| Model | Use when | Released | Signals | Status |
|---|---|---|---|---|
| MPT 30B | Use when the workload needs 8k context and 30B parameters. | 2023-03 | 8k context30B parameters | Current |
| MPT 7B | Use when the workload needs 2k context and 7B parameters. | 2023-03 | 2k context7B parameters | Current |
Release Timeline
1 release groupSpecifications(2 models)
Available From(2 providers)
Pricing
| Model | Provider | Input / 1M | Output / 1M | Type |
|---|---|---|---|---|
| MPT 7B | Databricks Foundation Model Serving | $0.5 | $0.5 | Serverless |
| MPT 30B | Databricks Foundation Model Serving | $1 | $1 | Serverless |
