Smaug
About
The Smaug large language model family by Abacus AI stands out for its exceptional performance in reasoning and mathematics. As open-source models, they are built on the foundation of Qwen-72B, enhanced through innovative fine-tuning techniques like DPO-Positive (DPOP), which tackles standard DPO loss limitations, especially in datasets with minimal differences between completion pairs. This advancement is documented in a research paper on arXiv. The Smaug-72B model has notably topped the Hugging Face Open LLM leaderboard, earning the distinction of being the first open-source model to score above 80 on average across key evaluations. These Smaug models are readily accessible on Hugging Face, fostering easy utilization and modification for the AI community at large 349.