Using Phind CodeLlama 34B V2 on DeepInfra

Quick Start

1. 1
   Create an account at DeepInfra and generate an API key.
2. 2
   Use the DeepInfra SDK or REST API to call phind-codellama-34b-v2 — see the documentation for request format.
3. 3
   You'll be billed $0.20/1M input, $0.45/1M output tokens. See full pricing.

Code Examples

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["DEEPINFRA_API_KEY"],
    base_url="https://api.deepinfra.com/v1/openai"
)
response = client.chat.completions.create(
    model="phind-codellama-34b-v2",
    messages=[{"role": "user", "content": "Hello"}]
)
print(response.choices[0].message.content)

About DeepInfra

DeepInfra offers serverless AI inference with a simple API, supporting hundreds of models across text generation, embeddings, and more. Pay-per-token pricing with no upfront commitments.

DeepInfra is a cloud inference platform offering cost-effective access to open-source AI models. It provides serverless inference for leading models from Meta, Mistral, Alibaba, and others with competitive token-based pricing.

Pricing on DeepInfra

Capabilities

About Phind CodeLlama 34B V2

Phind CodeLlama 34B v2 is a large language model designed specifically for code generation tasks, built on the CodeLlama architecture. It generates high-quality code in multiple programming languages such as Python, C/C++, TypeScript, and Java, and is instruction-tuned for enhanced usability. The model demonstrates strong benchmark performance, achieving a 73.8% pass@1 score on the HumanEval benchmark. It has been fine-tuned on a proprietary dataset of 1.5 billion tokens, focusing on instruction-answer pairs. Additionally, it showcases multi-lingual capabilities beyond programming languages and offers various quantized versions like GPTQ and GGUF for optimized performance and reduced memory usage. Despite its impressive features, thorough testing is advised prior to real-world deployment, as its testing has been limited so far 1 2 7.