Capybara
About
The Capybara family of large language models (LLMs), developed by Nous Research, is known for its cutting-edge training techniques and robust performance. These models excel in bilingual tasks, particularly in English and Chinese, thanks to their Yi-34B model foundation 5. A standout feature is their remarkable ability to handle extensive contexts of up to 200,000 tokens 25, which facilitates a profound understanding of complex topics and nuanced dialogues. The training approach incorporates an innovative synthesis method known as Amplify-Instruct, leveraging diverse techniques and instructional data from datasets like Airoboros, EverythingLM, and Know_Logic, as well as LessWrong posts 235. This training results in models capable of sophisticated reasoning, engaging in multi-turn dialogues, and effectively summarizing intricate subjects 235. The family encompasses models with parameter sizes of 3B, 7B, and 34B, with larger variants planned 23. Additionally, a multimodal extension named Obsidian enhances their range of functionalities 2.