Mamba
About
The Mamba family of large language models (LLMs) introduces a novel approach with its unique state space model (SSM) architecture 58. Diverging from traditional transformer models, Mamba uses selective SSMs to dynamically filter and interpret input content 58. This innovative method allows Mamba to efficiently process long sequences, achieving linear scalability in sequence length during both training and inference 58. Designed for hardware efficiency, it employs a parallel algorithm, akin to FlashAttention, to maximize GPU usage 58. Mamba has demonstrated exceptional performance on complex tasks such as language modeling, often equaling or surpassing transformer models of the same size 58. The Mamba architecture presents a promising alternative for applications demanding the handling of extended sequences 58.
Specifications(5 models)
| Model | Released | Parameters |
|---|---|---|
| Mamba 2.8B | 2023-12 | 2.8B |
| Mamba 1.4B | 2023-12 | 1.4B |
| Mamba 790M | 2023-12 | 790M |
| Mamba 370M | 2023-12 | — |
| Mamba 130M | 2023-12 | 130M |
Available From(1 provider)
Frequently Asked Questions
- What is Mamba?
- The Mamba family of large language models (LLMs) introduces a novel approach with its unique state space model (SSM) architecture 58. Diverging from traditional transformer models, Mamba uses selective SSMs to dynamically filter and interpret input content 58. This innovative method allows Mamba to efficiently process long sequences, achieving linear scalability in sequence length during both training and inference 58. Designed for hardware efficiency, it employs a parallel algorithm, akin to FlashAttention, to maximize GPU usage 58. Mamba has demonstrated exceptional performance on complex tasks such as language modeling, often equaling or surpassing transformer models of the same size 58. The Mamba architecture presents a promising alternative for applications demanding the handling of extended sequences 58.
- How many models are in the Mamba family?
- The Mamba family contains 5 models.
- What is the latest Mamba model?
- The latest model is Mamba 2.8B, released in 2023-12.
