Qwen-72B

The Qwen-72B is a potent large language model developed by Alibaba Cloud, featuring 72 billion parameters and built on the Transformer architecture. This model introduces enhancements such as SwiGLU activation, attention QKV bias, and group query attention for efficient performance on complex language tasks. Trained on a dataset of over 3 trillion tokens, it excels in multiple domains, including language understanding and generation, code generation, and translation, across multiple languages like Chinese and English. With an impressive context length of up to 32,000 tokens and a vocabulary exceeding 150,000 tokens, it manages extensive text inputs well. However, it bears limitations like inherited training data biases and lack of common sense reasoning. Despite these constraints, it remains a strong performer across various benchmarks, with a subsequent version, Qwen2-72B, offering further advancements.

Qwen-72B is a model in the Qwen family. The structured metadata tracks a 32k-token context window. This page tracks provider routes through Fireworks AI, with the cheapest tracked route listed at $0.9 input and $0.9 output per 1M tokens. No headline benchmark score is tracked for Qwen-72B yet.