What is the context window of DeepSeek MoE 16B?

DeepSeek MoE 16B has a context window of 4096 tokens.

When was DeepSeek MoE 16B released?

DeepSeek MoE 16B was released on 2024-01-11.

Name: DeepSeek MoE 16B
Author: DeepSeek

deepseek-moe-16b

Open Source

MoE variant with moderate parameter efficiency.

DeepSeek MoE 16B has a 4K-token context window.

VisionMultimodalReasoningFunction CallingTool UseStructured OutputsCode Execution

Released2024-01-11

Parameters16B

Context4K

ArchitectureMixture of Experts

Specializationgeneral

Trainingfinetuned

Fine-tuningbase

Advancing artificial general intelligence (AGI).

Hangzhou, Zhejiang, China

Founded 2023