RedPajama Models by Together.ai
About
The RedPajama family of large language models (LLMs) represents an open-source initiative focused on developing high-performing and transparent models, spearheaded by Together AI in collaboration with leading figures in the open-source AI community 83. These models are trained on the extensive RedPajama dataset, encompassing over 100 trillion raw tokens, and a refined subset of 30 trillion tokens across various languages and domains 8. They are available in multiple sizes and configurations, such as base models, instruction-tuned versions for enhanced few-shot learning, and chat models tailored for interactive dialogues 38. An exemplar model, the RedPajama-INCITE-Instruct-3B-v1, is particularly optimized for few-shot applications using GPT-JT data, deliberately excluding tasks overlapping with HELM core scenarios 3. The initiative not only prioritizes model performance but also the transparency and accessibility of data and training methodologies 8.
Current Variants
Use-when guidance is derived from seed capabilities, context, release, and replacement fields.
Use when the workload needs 2k context and 7B parameters.
Use when the workload needs 2k context and 3B parameters.
| Model | Use when | Released | Signals | Status |
|---|---|---|---|---|
| RedPajama INCITE 7B | Use when the workload needs 2k context and 7B parameters. | 2023-10 | 2k context7B parameters | Current |
| RedPajama INCITE 3B | Use when the workload needs 2k context and 3B parameters. | 2023-10 | 2k context3B parameters | Current |
Release Timeline
1 release groupSpecifications(2 models)
| Model | Released | Context | Parameters |
|---|---|---|---|
| RedPajama INCITE 7B | 2023-10 | 2k | 7b |
| RedPajama INCITE 3B | 2023-10 | 2k | 3b |
Frequently Asked Questions
- What is RedPajama used for?
- The RedPajama family of large language models (LLMs) represents an open-source initiative focused on developing high-performing and transparent models, spearheaded by Together AI in collaboration with leading figures in the open-source AI community 83.
- How does RedPajama compare to Together General?
- RedPajama by Together.ai is strongest where you need its listed use cases, while Together General by Together.ai is the closest related family to check for adjacent model selection. RedPajama has 2 listed variants and reaches up to 2k context, while Together General reaches up to 4k context, so compare the specs and pricing tables before choosing a production model.
- Which RedPajama model should I use?
- If price is the main constraint, use the pricing table first because RedPajama does not have complete provider pricing in the local data. For the most capable/latest local choice, evaluate RedPajama INCITE 7B with 2k context.
