Using CodeLlama 34B on DeepInfra

Quick Start

1. 1
   Create an account at DeepInfra and generate an API key.
2. 2
   Use the DeepInfra SDK or REST API to call codellama-34b — see the documentation for request format.
3. 3
   You'll be billed $0.20/1M input, $0.45/1M output tokens. See full pricing.

Code Examples

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["DEEPINFRA_API_KEY"],
    base_url="https://api.deepinfra.com/v1/openai"
)
response = client.chat.completions.create(
    model="codellama-34b",
    messages=[{"role": "user", "content": "Hello"}]
)
print(response.choices[0].message.content)

About DeepInfra

DeepInfra offers serverless AI inference with a simple API, supporting hundreds of models across text generation, embeddings, and more. Pay-per-token pricing with no upfront commitments.

DeepInfra is a cloud inference platform offering cost-effective access to open-source AI models. It provides serverless inference for leading models from Meta, Mistral, Alibaba, and others with competitive token-based pricing.

Pricing on DeepInfra

Capabilities

About CodeLlama 34B

CodeLlama 34B is a powerful generative text model developed by Meta, specifically tailored for code synthesis and understanding. With 34 billion parameters, it excels in code completion, infilling, and instruction following, particularly for Python programming. The model utilizes an auto-regressive transformer architecture and has been trained on a diverse dataset of programming languages, making it versatile for various coding tasks. Designed for both commercial and research applications, CodeLlama 34B offers AI engineers a robust tool for integrating advanced code generation capabilities into their projects. More details can be found on the model's Hugging Face page .