Falcon Mamba Models by Technology Innovation Institute (TII)
About
Falcon Mamba is TII's (Technology Innovation Institute) family of pure state-space language models (SSLMs) built on the Mamba architecture. The original Falcon Mamba 7B (August 2024) was the first competitive attention-free open-weight 7B model, outperforming similarly-sized transformers like Llama 3.1 8B on several benchmarks. Falcon3-Mamba-7B (December 2024) continued training on an additional 1.5 trillion tokens with higher-quality data, significantly improving reasoning and mathematical capabilities while maintaining the same architecture for backward compatibility. Both models use a 65K vocabulary (same as Mamba). The family demonstrates that pure SSM architectures can match transformer-based models at the 7B parameter scale.
Current Variants
Use-when guidance is derived from seed capabilities, context, release, and replacement fields.
Use when the workload needs 32K context and 7B parameters.
Use when the workload needs 8K context and 7B parameters.
| Model | Use when | Released | Signals | Status |
|---|---|---|---|---|
| Falcon3-Mamba-7B | Use when the workload needs 32K context and 7B parameters. | 2024-12 | 32K context7B parameters | Current |
| Falcon Mamba 7B | Use when the workload needs 8K context and 7B parameters. | 2024-08 | 8K context7B parameters | Current |
Release Timeline
2 release groupsSpecifications(2 models)
| Model | Released | Context | Parameters |
|---|---|---|---|
| Falcon3-Mamba-7B | 2024-12 | 32K | 7B |
| Falcon Mamba 7B | 2024-08 | 8K | 7B |
Frequently Asked Questions
- What is Falcon Mamba used for?
- Falcon Mamba is used for math-heavy prompts. The family description and listed model capabilities point to those workloads as the best fit.
- How does Falcon Mamba compare to Falcon?
- Falcon Mamba by Technology Innovation Institute (TII) is strongest where you need math-heavy prompts, while Falcon by Technology Innovation Institute (TII) is the closest related family to check for structured outputs. Falcon Mamba has 2 listed variants and reaches up to 32K context, while Falcon reaches up to 128K context, so compare the specs and pricing tables before choosing a production model.
- Which Falcon Mamba model should I use?
- If price is the main constraint, use the pricing table first because Falcon Mamba does not have complete provider pricing in the local data. For the most capable/latest local choice, evaluate Falcon3-Mamba-7B with 32K context.
