About
The Pangu large language model (LLM) family, developed by Huawei, comprises advanced models tailored for various natural language processing tasks. Launched in 2021 as pioneering pre-trained Chinese LLMs with over 100 billion parameters 7, Pangu models have seen remarkable evolution. Pangu-α, featuring 200 billion parameters, was adeptly trained on 2048 Ascend processors through a hybrid parallel training technique 3. The series also includes Pangu-π, which enhances model performance with architectural innovations for increased nonlinearity and expressive power 2, and is utilized in specialized applications such as the YunShan model for finance and law 2. Notably, Pangu-Σ, with a trillion parameters, utilizes a sparse heterogeneous computing architecture 8, while Pangu 3.0 targets industry-specific needs with a comprehensive three-layer structure 5. This LLM family exemplifies Huawei's dedication to producing robust, efficient models for varied uses, though details of training and parameter specifications might differ across sources.