Zamba 2 Models by Zyphra
Last refreshed 2026-05-19. Next refresh: weekly.
Details
About
The Zamba 2 family of large language models (LLMs) by Zyphra represents a novel integration of state-space Mamba and transformer blocks to achieve optimal performance, especially in low-resource settings. This architecture, built on a Mamba backbone, strategically alternates with transformer blocks to reduce parameters and conserve memory. Enhancements over the previous version include new Mamba2 blocks and dual interleaved attention layers, as well as LoRA projectors to tailor MLPs. The family offers models like the 2.7B and 7B ones, with the larger 7B version performing exceptionally well against peers of similar scale. Zamba2-7B-Instruct, a fine-tuned model variant, extends context length to 16,000 tokens, enhancing its prowess on instruction-following tasks. All models are open-source under the Apache 2.0 license, further promoting accessibility and innovation 235.
Decision facts
- Best fit
- General model comparison
- Capability starting point
- Zamba2 7B Instruct with 16k context
- Lowest tracked input
- Not tracked
- Closest related family
- ZAYA1
Current Variants
Use-when guidance is based on each model's tracked capabilities, context window, release date, and replacement status.
Use when the workload needs 4k context and 7B parameters.
Use when the workload needs 16k context and 7B parameters.
Use when the workload needs 4k context and 2.7B parameters.
Use when the workload needs 4k context and 2.7B parameters.
Use when the workload needs 4k context and 1.2B parameters.
Use when the workload needs 4k context and 1.2B parameters.
| Model | Use when | Released | Signals | Status |
|---|---|---|---|---|
| Zamba2 7B | Use when the workload needs 4k context and 7B parameters. | 2024-06 | 4k context7B parameters | Current |
| Zamba2 7B Instruct | Use when the workload needs 16k context and 7B parameters. | 2024-06 | 16k context7B parameters | Current |
| Zamba2 2.7B | Use when the workload needs 4k context and 2.7B parameters. | 2024-06 | 4k context2.7B parameters | Current |
| Zamba2 2.7B Instruct | Use when the workload needs 4k context and 2.7B parameters. | 2024-06 | 4k context2.7B parameters | Current |
| Zamba2 1.2B | Use when the workload needs 4k context and 1.2B parameters. | 2024-06 | 4k context1.2B parameters | Current |
| Zamba2 1.2B Instruct | Use when the workload needs 4k context and 1.2B parameters. | 2024-06 | 4k context1.2B parameters | Current |
Release Timeline
1 release groupSpecifications(6 models)
| Model | Released | Context | Parameters |
|---|---|---|---|
| Zamba2 7B | 2024-06 | 4k | 7B |
| Zamba2 7B Instruct | 2024-06 | 16k | 7B |
| Zamba2 2.7B | 2024-06 | 4k | 2.7B |
| Zamba2 2.7B Instruct | 2024-06 | 4k | 2.7B |
| Zamba2 1.2B | 2024-06 | 4k | 1.2B |
| Zamba2 1.2B Instruct | 2024-06 | 4k | 1.2B |
