Using OLMo 7B on Together AI

Quick Start

1. 1
   Create an account at Together AI and generate an API key.
2. 2
   Use the Together AI SDK or REST API to call olmo-7b — see the documentation for request format.
3. 3
   You'll be billed $0.20/1M input, $0.20/1M output tokens. See full pricing.

Code Examples

from together import Together

client = Together()  # reads TOGETHER_API_KEY from env
response = client.chat.completions.create(
    model="olmo-7b",
    messages=[{"role": "user", "content": "Hello"}]
)
print(response.choices[0].message.content)

About Together AI

Platform for running open-source and proprietary LLMs

Together AI is a platform for running open-source and proprietary LLMs with fast serverless and dedicated endpoints at competitive inference pricing.

Pricing on Together AI

Capabilities

About OLMo 7B

OLMo 7B is a large language model created by the Allen Institute for Artificial Intelligence (AI2), characterized by its open-source nature where model weights, training data, code, and evaluation tools have been publicly released. It utilizes a decoder-only transformer architecture, featuring 32 layers, a hidden size of 4096, and 32 attention heads, among other features. Trained on 2.5 trillion tokens from the Dolma dataset, this model excels in text generation, question answering, and language understanding, with performance metrics often comparable to or exceeding those of similar-sized models. It also boasts various architectural advancements such as SwiGLU activation functions and rotary positional embeddings. Despite its capabilities, users should be aware of its limitations concerning factual accuracy, bias, and context length.