LingoWhale
About
The LingoWhale family of large language models (LLMs) is developed by DeepLangAI in collaboration with THUNLP Lab. This series includes LingoWhale-8B, a bilingual model pre-trained on a substantial volume of high-quality Chinese-English data 1. With an 8K context window, it excels in comprehending and generating longer sequences. The model is open-source for academic research, though commercial use requires prior approval 1. LingoWhale-8B achieves high performance on public benchmarks but faces challenges such as hallucinations and weaker mathematical capabilities. Subsequent versions aim to overcome these limitations, with support provided through model weights, a Huggingface inference interface, and parameter-efficient fine-tuning examples like LoRA 1.