Striped Hyena
About
The Striped Hyena family of large language models (LLMs), developed by Together AI and collaborators, offers an innovative approach to model architecture with its hybrid design. This family integrates multi-head, grouped-query attention and gated convolutions within Hyena blocks, which distinguishes it from conventional transformer models. This unique structure enhances training and inference capabilities, particularly for handling long sequences. The family encompasses both base models like the StripedHyena-Hessian-7B (SH 7B) and chat models such as StripedHyena-Nous-7B (SH-N 7B), all optimized for efficiency with faster processing speeds and reduced memory usage during autoregressive tasks. Designed to handle prompt lengths up to 128,000 tokens, Striped Hyena models deliver performance competitive with top open-source transformers on a variety of benchmarks 46.
Specifications(2 models)
| Model | Released | Parameters | Structured Outputs |
|---|---|---|---|
| StripedHyena Hessian 7B | 2023-12 | 7B | Yes |
| StripedHyena Nous 7B | 2023-12 | 7B | Yes |
Available From(1 provider)
Pricing
| Model | Provider | Input / 1M | Output / 1M | Type |
|---|---|---|---|---|
| StripedHyena Nous 7B | Together AI | $0.2 | $0.2 | Serverless |
| StripedHyena Hessian 7B | Together AI | $0.2 | $0.2 | Serverless |
Frequently Asked Questions
- What is Striped Hyena?
- The Striped Hyena family of large language models (LLMs), developed by Together AI and collaborators, offers an innovative approach to model architecture with its hybrid design. This family integrates multi-head, grouped-query attention and gated convolutions within Hyena blocks, which distinguishes it from conventional transformer models. This unique structure enhances training and inference capabilities, particularly for handling long sequences. The family encompasses both base models like the StripedHyena-Hessian-7B (SH 7B) and chat models such as StripedHyena-Nous-7B (SH-N 7B), all optimized for efficiency with faster processing speeds and reduced memory usage during autoregressive tasks. Designed to handle prompt lengths up to 128,000 tokens, Striped Hyena models deliver performance competitive with top open-source transformers on a variety of benchmarks 46.
- How many models are in the Striped Hyena family?
- The Striped Hyena family contains 2 models.
- What is the latest Striped Hyena model?
- The latest model is StripedHyena Hessian 7B, released in 2023-12.
- How much does Striped Hyena cost?
- Striped Hyena models are available at $0.2/1M input tokens through providers like Together AI.
