LIVE

Amazon Bedrock Adds Prompt Caching Support for Open-Weight Models

🔌 Chips & Hardware 2 min read 👁 6

Amazon Web Services has introduced prompt caching capabilities for open-weight models hosted on Amazon Bedrock, designed to lower input expenses and decrease response latency.

Amazon Bedrock Adds Prompt Caching Support for Open-Weight Models🔌Chips & Hardware

Photo: AWS Machine Learning Blog

Key points

  • Amazon Bedrock now supports implicit and explicit prompt caching for open-weight models.
  • The feature reduces input costs and response latency by reusing previously processed context prefixes.
  • Cache reads are billed at a lower rate, while data privacy protections prevent prompts from being used for model training.

Amazon Web Services has rolled out prompt caching support for open-weight models available on Amazon Bedrock. The functionality allows generative AI applications to reuse long contexts—such as system prompts, reference materials, tool descriptions, and ongoing conversation histories—without processing the same data repeatedly.

According to the company, the feature operates through two distinct modes depending on the model and API configuration: implicit caching and explicit caching. Implicit caching works automatically on a best-effort basis as conversation turns progress, whereas explicit caching allows developers to designate specific cache breakpoints using the Chat Completions API. Cache reads are billed at reduced rates compared to standard inputs, though cache writes may carry an increased fee depending on the model.

AWS notes that cached data remains protected under its standard data privacy policies, meaning user prompts and outputs are not shared with model providers or used for training purposes. Developers utilizing the system can structure static context at the beginning of prompts to maximize cache hits and optimize overall operational costs.

What to watch next: Monitor adoption rates among developers utilizing open-weight architectures on AWS and any updates to pricing structures for cache reads and writes.

Source AWS Machine Learning Blog — aws.amazon.com/blogs/machine-learning/use-prompt-caching-with-open-weig… ↗

Contact the newsroom

Got a tip, a proposal or a partnership question? Write to us and we will reply shortly.