DeepSeek MoE Models by DeepSeek
Details
About
The DeepSeek MoE family represents a groundbreaking series of large language models that leverage the Mixture-of-Experts (MoE) architecture to optimize computational efficiency and performance. By selectively activating a subset of parameters, known as "experts," these models minimize computational demands while maintaining robust capabilities. They utilize strategies like fine-grained expert segmentation, allowing specialized unit divisions, and shared expert isolation, which reduces redundancy in common knowledge experts. The lineup includes models such as the DeepSeekMoE 16B, which matches the performance of LLaMA2 7B with significantly less computation, and the DeepSeek-V2 family, incorporating Multi-head Latent Attention (MLA) for enhanced inference efficiency. Available on platforms like Hugging Face and GitHub, DeepSeek models span a range from the extensive DeepSeek-V2, with 236 billion parameters, to the more compact DeepSeek-V2-Lite, facilitating open-source research and development.
Current Variants
Use-when guidance is derived from seed capabilities, context, release, and replacement fields.
Use when the workload needs 4k context and 16B parameters.
Use when the workload needs 4k context and 16B parameters.
| Model | Use when | Released | Signals | Status |
|---|---|---|---|---|
| DeepSeek MoE 16B | Use when the workload needs 4k context and 16B parameters. | 2024-01 | 4k context16B parameters | Current |
| DeepSeek MoE 16B Chat | Use when the workload needs 4k context and 16B parameters. | 2024-01 | 4k context16B parameters | Current |
Release Timeline
1 release groupSpecifications(2 models)
| Model | Released | Context | Parameters |
|---|---|---|---|
| DeepSeek MoE 16B | 2024-01 | 4k | 16B |
| DeepSeek MoE 16B Chat | 2024-01 | 4k | 16B |
Frequently Asked Questions
- What is DeepSeek MoE used for?
- The DeepSeek MoE family represents a groundbreaking series of large language models that leverage the Mixture-of-Experts (MoE) architecture to optimize computational efficiency and performance.
- How does DeepSeek MoE compare to Janus?
- DeepSeek MoE by DeepSeek is strongest where you need its listed use cases, while Janus by DeepSeek is the closest related family to check for image generation. DeepSeek MoE has 2 listed variants and reaches up to 4k context, so compare the specs and pricing tables before choosing a production model.
- Which DeepSeek MoE model should I use?
- If price is the main constraint, use the pricing table first because DeepSeek MoE does not have complete provider pricing in the local data. For the most capable/latest local choice, evaluate DeepSeek MoE 16B with 4k context.
