Baichuan 13B Chat

Baichuan-13B-Chat is a 13-billion parameter large language model from Baichuan Intelligent Technology, building upon their earlier Baichuan-7B model. It excels in natural language processing tasks for both Chinese and English languages. Notable features include a substantial training dataset encompassing 1.4 trillion tokens—40% more than the LLaMA-13B model—and superior dialogue capabilities. The model efficiently operates on consumer-grade GPUs, such as the Nvidia 3090, thanks to its efficient inference. Additionally, it employs ALiBi positional encoding with a context window of 4096 tokens. Baichuan-13B-Chat is open-source with commercial usability under the appropriate licensing and is based on the transformer architecture, providing details on hidden size, layers, and attention heads in its documentation. Its performance on various benchmarks is impressive, surpassing other models of similar size.

Baichuan 13B Chat is a model in the Baichuan family. The structured metadata tracks a 4k-token context window. No headline benchmark score is tracked for Baichuan 13B Chat yet.