StripedHyena Nous 7B

StripedHyena-Nous-7B (SH-N 7B) is a state-of-the-art large language AI model from Together Computer, developed alongside Nous Research. Diverging from the traditional Transformer-based architecture, SH-N 7B employs a unique design integrating multi-head, grouped-query attention with gated convolutions in structured Hyena blocks. This hybrid architecture enhances its capacity for long-context processing and offers superior training efficiency and decoding speeds. The model is adept in chat applications, capable of engaging in coherent long-form dialogues, answering questions, and performing various language tasks. Despite requiring specific hardware configurations, SH-N 7B presents competitive performance comparable to leading open-source Transformer models. It’s trained on extensive datasets, including RedPajama, optimized for both short and long-context sequences up to 32k tokens.

StripedHyena Nous 7B is an open-source model in the Striped Hyena family. The structured metadata tracks a 32k-token context window and structured outputs. This page tracks provider routes through Together AI, with the cheapest tracked route listed at $0.2 input and $0.2 output per 1M tokens. No headline benchmark score is tracked for StripedHyena Nous 7B yet.