Falcon Mamba 7B
Falcon Mamba 7B is released 2024-08-12 in the Falcon Mamba family with open-weight; evaluate it while provider pricing coverage matures.
Use it for
- Teams evaluating general LLM work
- Workloads that can use a 8k context window
Do not use it for
- Cost-sensitive launches that need sourced token pricing
- Vision or document-understanding workloads
- Strict JSON or tool-calling flows
- Family
- Falcon Mamba
- Released
- 2024-08-12
- Context
- 8k
- Parameters
- 7B
- Architecture
- Decoder Only
- Specialization
- general
- Openness
- Open weights
- License
- TII Falcon License 2.0Commercial use: permitted
- Weights
- Available
- Code
- Unknown
- Training
- Pretrained
No tracked provider token pricing is available yet.
About
Falcon Mamba 7B is TII's first open-weight state-space language model (SSLM), released August 12, 2024. The first competitive attention-free 7B model, verified by HuggingFace as the no.1 open-source SSLM globally at release. Built on the Mamba architecture with extra RMS normalization layers for stable training at scale. Outperforms Llama 3.1 8B and Mistral 7B on several benchmarks despite using no attention. 65K vocabulary, 64 layers, 4096 hidden dimension. Trained on approximately 5.5 trillion tokens from RefinedWeb and Fineweb-edu. Succeeded by Falcon3-Mamba-7B (December 2024).
Falcon Mamba 7B is an open-weight model in the Falcon Mamba family. The structured metadata tracks a 8k-token context window. No headline benchmark score is tracked for Falcon Mamba 7B yet.
Top use-case fit
No primary decision-task fit is mapped for this model yet.
Provider price ladder
No tracked provider token pricing is available for this model yet.
Capabilities
No model capability flags are currently sourced.
Benchmark peer barsfor Coding
No task-mapped benchmark peers are available for this model yet.
Migration checks
No linked migration route is available for this model yet.
Frequently asked questions
What is the context window of Falcon Mamba 7B?
Falcon Mamba 7B has a context window of 8k tokens.
When was Falcon Mamba 7B released?
Falcon Mamba 7B was released on 2024-08-12.
No tracked provider token pricing is available yet.