StripedHyena Nous 7B

About

StripedHyena-Nous-7B (SH-N 7B) is a state-of-the-art large language AI model from Together Computer, developed alongside Nous Research. Diverging from the traditional Transformer-based architecture, SH-N 7B employs a unique design integrating multi-head, grouped-query attention with gated convolutions in structured Hyena blocks. This hybrid architecture enhances its capacity for long-context processing and offers superior training efficiency and decoding speeds. The model is adept in chat applications, capable of engaging in coherent long-form dialogues, answering questions, and performing various language tasks. Despite requiring specific hardware configurations, SH-N 7B presents competitive performance comparable to leading open-source Transformer models. It’s trained on extensive datasets, including RedPajama, optimized for both short and long-context sequences up to 32k tokens.

Capabilities

Providers(1)