Using Llama 3.1 8B Instruct on DeepInfra

Quick Start

1. 1
   Create an account at DeepInfra and generate an API key.
2. 2
   Use the DeepInfra SDK or REST API to call meta-llama/Meta-Llama-3.1-8B-Instruct — see the documentation for request format.
3. 3
   You'll be billed $0.02/1M input, $0.05/1M output tokens. See full pricing.

Code Examples

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["DEEPINFRA_API_KEY"],
    base_url="https://api.deepinfra.com/v1/openai"
)
response = client.chat.completions.create(
    model="meta-llama/Meta-Llama-3.1-8B-Instruct",
    messages=[{"role": "user", "content": "Hello"}]
)
print(response.choices[0].message.content)

About DeepInfra

DeepInfra offers serverless AI inference with a simple API, supporting hundreds of models across text generation, embeddings, and more. Pay-per-token pricing with no upfront commitments.

DeepInfra is a cloud inference platform offering cost-effective access to open-source AI models. It provides serverless inference for leading models from Meta, Mistral, Alibaba, and others with competitive token-based pricing.

Pricing on DeepInfra

Capabilities

About Llama 3.1 8B Instruct

The Llama 3.1 8B Instruct model, released on July 23, 2024, is a multilingual large language model with 8 billion parameters, optimized for instruction-following tasks. It features an enhanced transformer architecture, supporting languages like English, German, French, and others. The model excels in dialogue applications, having been fine-tuned using supervised fine-tuning and reinforcement learning with human feedback. Trained on approximately 15 trillion tokens with a December 2023 data cutoff, it outperforms many existing open-source and closed chat models in various benchmarks. Ideal for commercial and research applications such as conversational agents and content generation, the model can be accessed on Hugging Face .