InternLM2
About
InternLM2 is an open-source large language model (LLM) family developed collaboratively by researchers from the Shanghai AI Laboratory, SenseTime Group, The Chinese University of Hong Kong, and Fudan University 157. The InternLM2 models excel in multiple benchmarks, showcasing their superior capabilities and enhanced performance over predecessors 15. These advancements are attributed to innovative pre-training and optimization techniques, emphasizing data preparation involving text, code, and extensive long-context data 1. The training regimen begins with 4k tokens, expanding to 32k tokens through successive pre-training and fine-tuning phases 1. A standout feature is the application of COOL RLHF strategy to address varied human feedback preferences and prevent reward manipulation 1. Models in the InternLM2 family vary in size, with parameters ranging from 1.8B to 20B, offering different training stage releases for community evaluation 146. Notably, they perform exceptionally well on the "Needle-in-a-Haystack" test, highlighting robust long-context processing capabilities 1. Specific variants, such as InternLM2-Math-Plus, demonstrate state-of-the-art proficiency in mathematical reasoning 4.