StarCoder 2 Models by ServiceNow Research
About
StarCoder 2 is an advanced family of open large language models (LLMs) adept at code generation, developed collaboratively by BigCode, Hugging Face, and NVIDIA. This model family includes variants with 3 billion, 7 billion, and 15 billion parameters. Trained on The Stack v2, a colossal dataset over 4 trillion tokens and more than 600 programming languages, StarCoder 2 is built for high performance. It leverages innovative techniques like Grouped Query Attention, a 16,384-token context window, and a 4,096-token sliding window attention, employing the Fill-in-the-Middle objective during training. The StarCoder2-15B model, in particular, showcases exceptional capabilities, often matching or surpassing larger models. Available under the BigCode OpenRAIL-M v1 license, StarCoder 2 offers extensive potential for complex code-based applications134.
Current Variants
Use-when guidance is derived from seed capabilities, context, release, and replacement fields.
Use when the workload needs 8k context and 7B parameters.
Use when the workload needs 8k context and 3B parameters.
Use when the workload needs 16k context, 15B parameters, and structured outputs.
| Model | Use when | Released | Signals | Status |
|---|---|---|---|---|
| StarCoder2 7B | Use when the workload needs 8k context and 7B parameters. | 2024-07 | 8k context7B parameters | Current |
| StarCoder2 3B | Use when the workload needs 8k context and 3B parameters. | 2024-07 | 8k context3B parameters | Current |
| DeepInfra StarCoder2 15B | Use when the workload needs 16k context, 15B parameters, and structured outputs. | 2024-07 | 16k context15B parametersstructured outputs | Current |
Release Timeline
1 release groupSpecifications(4 models)
| Model | Released | Context | Parameters | Structured Outputs |
|---|---|---|---|---|
| StarCoder2 7B | 2024-07 | 8k | 7B | No |
| StarCoder2 3B | 2024-07 | 8k | 3B | No |
| DeepInfra StarCoder2 15B | 2024-07 | 16k | 15B | Yes |
Available From(3 providers)
Pricing
| Model | Provider | Input / 1M | Output / 1M | Type |
|---|---|---|---|---|
| StarCoder2 3B | Fireworks AI | $0.1 | $0.1 | Provisioned |
| StarCoder2 7B | Fireworks AI | $0.2 | $0.2 | Serverless |
| DeepInfra StarCoder2 15B | DeepInfra | $0.2 | $0.6 | Serverless |
Frequently Asked Questions
- What is StarCoder 2 used for?
- StarCoder 2 is used for coding and structured outputs. The family description and listed model capabilities point to those workloads as the best fit.
- How does StarCoder 2 compare to StarCoder?
- StarCoder 2 by ServiceNow Research is strongest where you need coding, while StarCoder by ServiceNow Research is the closest related family to check for coding. StarCoder 2 has 4 listed variants and reaches up to 16k context, while StarCoder reaches up to 8k context, so compare the specs and pricing tables before choosing a production model.
- Which StarCoder 2 model should I use?
- For the lowest listed input price, start with StarCoder2 3B through Fireworks AI at $0.1/1M input tokens. For the most capable/latest local choice, evaluate DeepInfra StarCoder2 15B with 16k context and structured outputs.
