Falcon Mamba 7B
Falcon Mamba 7B is released 2024-08-12 in the Falcon Mamba family with open-weight; evaluate it while provider pricing coverage matures.
- Family
- Falcon Mamba
- Released
- 2024-08-12
- Context
- 8k
- Parameters
- 7B
- Architecture
- Decoder Only
- Specialization
- general
- Openness
- Open weights
- License
- TII Falcon License 2.0Commercial use: permitted
- Weights
- Available
- Code
- Unknown
- Training
- Pretrained
No tracked provider token pricing is available yet.
About
Falcon Mamba 7B is TII's first open-weight state-space language model (SSLM), released August 12, 2024. The first competitive attention-free 7B model, verified by HuggingFace as the no.1 open-source SSLM globally at release. Built on the Mamba architecture with extra RMS normalization layers for stable training at scale. Outperforms Llama 3.1 8B and Mistral 7B on several benchmarks despite using no attention. 65K vocabulary, 64 layers, 4096 hidden dimension. Trained on approximately 5.5 trillion tokens from RefinedWeb and Fineweb-edu. Succeeded by Falcon3-Mamba-7B (December 2024).
Provider price ladder
No tracked provider token pricing is available for this model yet.
Capabilities
No model capability flags are currently sourced.
Benchmark peer barsfor Coding
No task-mapped benchmark peers are available for this model yet.
Migration checks
No linked migration route is available for this model yet.
No tracked provider token pricing is available yet.