Falcon Mamba Models by Technology Innovation Institute (TII)
Last refreshed 2026-05-25. Next refresh: weekly.
Details
About
Falcon Mamba is TII's (Technology Innovation Institute) family of pure state-space language models (SSLMs) built on the Mamba architecture. The original Falcon Mamba 7B (August 2024) was the first competitive attention-free open-weight 7B model, outperforming similarly-sized transformers like Llama 3.1 8B on several benchmarks. Falcon3-Mamba-7B (December 2024) continued training on an additional 1.5 trillion tokens with higher-quality data, significantly improving reasoning and mathematical capabilities while maintaining the same architecture for backward compatibility. Both models use a 65K vocabulary (same as Mamba). The family demonstrates that pure SSM architectures can match transformer-based models at the 7B parameter scale.
Decision facts
- Best fit
- math-heavy prompts
- Capability starting point
- Falcon3-Mamba-7B with 32k context
- Lowest tracked input
- Not tracked
- Closest related family
- Falcon
Current Variants
Use-when guidance is based on each model's tracked capabilities, context window, release date, and replacement status.
Use when the workload needs 32k context and 7B parameters.
Use when the workload needs 8k context and 7B parameters.
| Model | Use when | Released | Signals | Status |
|---|---|---|---|---|
| Falcon3-Mamba-7B | Use when the workload needs 32k context and 7B parameters. | 2024-12 | 32k context7B parameters | Current |
| Falcon Mamba 7B | Use when the workload needs 8k context and 7B parameters. | 2024-08 | 8k context7B parameters | Current |
Release Timeline
2 release groupsSpecifications(2 models)
| Model | Released | Context | Parameters |
|---|---|---|---|
| Falcon3-Mamba-7B | 2024-12 | 32k | 7B |
| Falcon Mamba 7B | 2024-08 | 8k | 7B |
