Aquila 2 70B Expressive
About
The Aquila2-70B-Expr, developed by the Beijing Academy of Artificial Intelligence, is a large language model featuring 70 billion parameters. It is part of the Aquila2 series, known for its bilingual capabilities in English and Chinese. This experimental model excels in language understanding and generation, supported by the HeuriMentor framework which optimizes training through dynamic data distribution. Its training process benefits from techniques like Grouped Query Attention and Rotary Position Embedding for improved efficiency. As an open-source model, Aquila2-70B-Expr's weights and code are accessible, allowing independent research and development, though users should note its experimental nature and the substantial computational resources required for its deployment.