InternLM2 Models by Intern-AI
About
InternLM2 is an open-source large language model (LLM) family developed collaboratively by researchers from the Shanghai AI Laboratory, SenseTime Group, The Chinese University of Hong Kong, and Fudan University 157. The InternLM2 models excel in multiple benchmarks, showcasing their superior capabilities and enhanced performance over predecessors 15. These advancements are attributed to innovative pre-training and optimization techniques, emphasizing data preparation involving text, code, and extensive long-context data 1. The training regimen begins with 4k tokens, expanding to 32k tokens through successive pre-training and fine-tuning phases 1. A standout feature is the application of COOL RLHF strategy to address varied human feedback preferences and prevent reward manipulation 1. Models in the InternLM2 family vary in size, with parameters ranging from 1.8B to 20B, offering different training stage releases for community evaluation 146. Notably, they perform exceptionally well on the "Needle-in-a-Haystack" test, highlighting robust long-context processing capabilities 1. Specific variants, such as InternLM2-Math-Plus, demonstrate state-of-the-art proficiency in mathematical reasoning 4.
Current Variants
Use-when guidance is derived from seed capabilities, context, release, and replacement fields.
Use when the workload needs 131k context and 7 parameters.
Use when the workload needs 131k context and 20 parameters.
| Model | Use when | Released | Signals | Status |
|---|---|---|---|---|
| InternLM 2 Turbo | Use when the workload needs 262k context. | 2024-06 | 262k context | Current |
| InternLM 2 7B | Use when the workload needs 131k context and 7 parameters. | 2024-01 | 131k context7 parameters | Current |
| InternLM 2 20B | Use when the workload needs 131k context and 20 parameters. | 2024-01 | 131k context20 parameters | Current |
| InternLM 2 Chat | Use when the workload needs 131k context. | 2024-01 | 131k context | Current |
| InternLM2 20B | Use when the workload needs 20B parameters. | 2024-01 | 20B parameters | Current |
| InternLM2 7B | Use when the workload needs 7B parameters. | 2024-01 | 7B parameters | Current |
| InternLM2 1.8B | Use when the workload needs 1.8B parameters. | 2024-01 | 1.8B parameters | Current |
Release Timeline
2 release groupsSpecifications(7 models)
| Model | Released | Context | Parameters |
|---|---|---|---|
| InternLM 2 Turbo | 2024-06 | 262k | — |
| InternLM 2 7B | 2024-01 | 131k | 7 |
| InternLM 2 20B | 2024-01 | 131k | 20 |
| InternLM 2 Chat | 2024-01 | 131k | — |
| InternLM2 20B | 2024-01 | — | 20B |
| InternLM2 7B | 2024-01 | — | 7B |
| InternLM2 1.8B | 2024-01 | — | 1.8B |
Frequently Asked Questions
- What is InternLM2 used for?
- InternLM2 is used for coding and math-heavy prompts. The family description and listed model capabilities point to those workloads as the best fit.
- How does InternLM2 compare to InternVL?
- InternLM2 by Intern-AI is strongest where you need coding, while InternVL by Intern-AI is the closest related family to check for adjacent model selection. InternLM2 has 7 listed variants and reaches up to 262k context, while InternVL reaches up to 32k context, so compare the specs and pricing tables before choosing a production model.
- Which InternLM2 model should I use?
- If price is the main constraint, use the pricing table first because InternLM2 does not have complete provider pricing in the local data. For the most capable/latest local choice, evaluate InternLM 2 Turbo with 262k context.
