Using Llama 3 70B Instruct on AWS Bedrock

Quick Start

1. 1
   Create an account at AWS Bedrock and generate an API key.
2. 2
   Use the AWS Bedrock SDK or REST API to call llama3-70b-instruct — see the documentation for request format.
3. 3
   You'll be billed $0.99/1M input, $0.99/1M output tokens. See full pricing.

Code Examples

import boto3

# Reads AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_DEFAULT_REGION from env
client = boto3.client("bedrock-runtime", region_name="us-east-1")
response = client.converse(
    modelId="llama3-70b-instruct",
    messages=[{
        "role": "user",
        "content": [{"text": "Hello"}]
    }]
)
print(response["output"]["message"]["content"][0]["text"])

About AWS Bedrock

Amazon Bedrock is a comprehensive, fully managed service for building and scaling generative AI applications. The platform provides access to a diverse array of high-performing foundation models (FMs) from leading AI companies through a unified API, enabling users to select the most suitable models for their specific use cases. Key features include model customization using proprietary data through techniques like fine-tuning and Retrieval Augmented Generation (RAG), which significantly enhances the relevance and accuracy of AI outputs. Additionally, the platform supports the automation of complex tasks with agents capable of executing multi-step operations, making it versatile for applications ranging from text generation and image creation to conversational AI. Beyond its robust technical capabilities, Amazon Bedrock offers a serverless experience that streamlines infrastructure management, allowing developers to focus on application development without the burden of managing underlying resources. The platform prioritizes security and compliance, ensuring that data remains within the AWS ecosystem and adheres to industry standards. Bedrock's flexible pricing models, including pay-as-you-go options, enable organizations to effectively manage costs while scaling their AI initiatives. This combination of advanced features, ease of use, and cost-effectiveness positions Amazon Bedrock as a powerful tool for businesses looking to innovate rapidly in the generative AI space, ultimately enhancing productivity and operational efficiency.

AWS Bedrock is Amazon's fully managed foundation-model service, providing unified API access to top models from Anthropic, Meta, Mistral, and other leading AI labs with built-in tools for RAG, fine-tuning, and AI agent development.

Pricing on AWS Bedrock

Capabilities

About Llama 3 70B Instruct

The Llama 3 70B Instruct model is a large language model with 70 billion parameters, released by Meta on April 18, 2024. It's an instruction-tuned variant optimized for conversational applications, utilizing an advanced auto-regressive transformer architecture. The model excels in following instructions and engaging in dialogue, having been trained on over 15 trillion tokens with a December 2023 knowledge cutoff. It demonstrates superior performance on industry benchmarks, scoring 82.0 on the MMLU (5-shot) test. The model incorporates extensive safety measures and optimizations, including RLHF, to enhance helpfulness and reduce harmful content generation. For more details, visit the model's Hugging Face page [1].