Using Llama 3.1 70B Instruct on AWS Bedrock

Quick Start

1. 1
   Create an account at AWS Bedrock and generate an API key.
2. 2
   Use the AWS Bedrock SDK or REST API to call llama3.1-70b-instruct — see the documentation for request format.
3. 3
   You'll be billed $0.72/1M input, $0.72/1M output tokens. See full pricing.

Code Examples

import boto3

# Reads AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_DEFAULT_REGION from env
client = boto3.client("bedrock-runtime", region_name="us-east-1")
response = client.converse(
    modelId="llama3.1-70b-instruct",
    messages=[{
        "role": "user",
        "content": [{"text": "Hello"}]
    }]
)
print(response["output"]["message"]["content"][0]["text"])

About AWS Bedrock

Amazon Bedrock is a comprehensive, fully managed service for building and scaling generative AI applications. The platform provides access to a diverse array of high-performing foundation models (FMs) from leading AI companies through a unified API, enabling users to select the most suitable models for their specific use cases. Key features include model customization using proprietary data through techniques like fine-tuning and Retrieval Augmented Generation (RAG), which significantly enhances the relevance and accuracy of AI outputs. Additionally, the platform supports the automation of complex tasks with agents capable of executing multi-step operations, making it versatile for applications ranging from text generation and image creation to conversational AI. Beyond its robust technical capabilities, Amazon Bedrock offers a serverless experience that streamlines infrastructure management, allowing developers to focus on application development without the burden of managing underlying resources. The platform prioritizes security and compliance, ensuring that data remains within the AWS ecosystem and adheres to industry standards. Bedrock's flexible pricing models, including pay-as-you-go options, enable organizations to effectively manage costs while scaling their AI initiatives. This combination of advanced features, ease of use, and cost-effectiveness positions Amazon Bedrock as a powerful tool for businesses looking to innovate rapidly in the generative AI space, ultimately enhancing productivity and operational efficiency.

AWS Bedrock is Amazon's fully managed foundation-model service, providing unified API access to top models from Anthropic, Meta, Mistral, and other leading AI labs with built-in tools for RAG, fine-tuning, and AI agent development.

Pricing on AWS Bedrock

Capabilities

About Llama 3.1 70B Instruct

The Llama 3.1 70B Instruct model is a cutting-edge large language model with 70 billion parameters, designed for instruction-following tasks. It features multilingual capabilities, supporting languages like English, German, French, and others. Fine-tuned using supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF), it excels in understanding and responding to user instructions. The model can handle a context length of up to 128k tokens, making it suitable for complex dialogue systems and applications requiring detailed responses. It outperforms many existing open-source and proprietary models on various industry benchmarks, making it ideal for conversational AI, content generation, and data synthesis tasks. For more details, visit the Hugging Face page [1].