StarCoder 2
About
StarCoder 2 is an advanced family of open large language models (LLMs) adept at code generation, developed collaboratively by BigCode, Hugging Face, and NVIDIA. This model family includes variants with 3 billion, 7 billion, and 15 billion parameters. Trained on The Stack v2, a colossal dataset over 4 trillion tokens and more than 600 programming languages, StarCoder 2 is built for high performance. It leverages innovative techniques like Grouped Query Attention, a 16,384-token context window, and a 4,096-token sliding window attention, employing the Fill-in-the-Middle objective during training. The StarCoder2-15B model, in particular, showcases exceptional capabilities, often matching or surpassing larger models. Available under the BigCode OpenRAIL-M v1 license, StarCoder 2 offers extensive potential for complex code-based applications134.
Specifications(3 models)
| Model | Released | Context | Parameters | Structured Outputs |
|---|---|---|---|---|
| StarCoder2 7B | 2024-07 | 8K | 7B | No |
| StarCoder2 3B | 2024-07 | 8K | 3B | No |
Available From(3 providers)
Pricing
| Model | Provider | Input / 1M | Output / 1M | Type |
|---|---|---|---|---|
| StarCoder2 3B | Fireworks AI | $0.1 | $0.1 | Provisioned |
| StarCoder2 7B | Fireworks AI | $0.2 | $0.2 | Serverless |
Frequently Asked Questions
- What is StarCoder 2?
- StarCoder 2 is an advanced family of open large language models (LLMs) adept at code generation, developed collaboratively by BigCode, Hugging Face, and NVIDIA. This model family includes variants with 3 billion, 7 billion, and 15 billion parameters. Trained on The Stack v2, a colossal dataset over 4 trillion tokens and more than 600 programming languages, StarCoder 2 is built for high performance. It leverages innovative techniques like Grouped Query Attention, a 16,384-token context window, and a 4,096-token sliding window attention, employing the Fill-in-the-Middle objective during training. The StarCoder2-15B model, in particular, showcases exceptional capabilities, often matching or surpassing larger models. Available under the BigCode OpenRAIL-M v1 license, StarCoder 2 offers extensive potential for complex code-based applications134.
- How many models are in the StarCoder 2 family?
- The StarCoder 2 family contains 3 models.
- What is the latest StarCoder 2 model?
- The latest model is StarCoder2 15B, released in 2024-07.
- How much does StarCoder 2 cost?
- StarCoder 2 models range from $0.1/1M to $0.2/1M input tokens depending on the model and provider.
