Amazon Bedrock Adds Prompt Caching Support for Open-Weight Models
Amazon Web Services has introduced prompt caching capabilities for open-weight models hosted on Amazon Bedrock, designed to lower input expenses and decrease response latency.
🔌Chips & HardwarePhoto: AWS Machine Learning Blog
Key points
- Amazon Bedrock now supports implicit and explicit prompt caching for open-weight models.
- The feature reduces input costs and response latency by reusing previously processed context prefixes.
- Cache reads are billed at a lower rate, while data privacy protections prevent prompts from being used for model training.
Amazon Web Services has rolled out prompt caching support for open-weight models available on Amazon Bedrock. The functionality allows generative AI applications to reuse long contexts—such as system prompts, reference materials, tool descriptions, and ongoing conversation histories—without processing the same data repeatedly.
According to the company, the feature operates through two distinct modes depending on the model and API configuration: implicit caching and explicit caching. Implicit caching works automatically on a best-effort basis as conversation turns progress, whereas explicit caching allows developers to designate specific cache breakpoints using the Chat Completions API. Cache reads are billed at reduced rates compared to standard inputs, though cache writes may carry an increased fee depending on the model.
AWS notes that cached data remains protected under its standard data privacy policies, meaning user prompts and outputs are not shared with model providers or used for training purposes. Developers utilizing the system can structure static context at the beginning of prompts to maximize cache hits and optimize overall operational costs.
What to watch next: Monitor adoption rates among developers utilizing open-weight architectures on AWS and any updates to pricing structures for cache reads and writes.
Read next
🧠AI AnalysisPedestal AI Secures Seed Funding to Automate Logistics With Intelligent Agents
Pedestal AI has launched publicly with $4.7 million in seed backing to deploy agentic artificial intelligence aimed at streamlining logistics and supply chain operations.
🏢Big TechUnited Launch Alliance Investigates Vulcan Rocket Failure
United Launch Alliance is investigating a mission failure of its Vulcan rocket, which suffered from low booster performance shortly after liftoff. The incident raises que…
🌐Work & SocietyUS Southwest Hydropower Output Hits Record Lows Amid Persistent Drought
Hydropower generation in the US Southwest dropped 28% below the 10-year average this summer as record-low water levels in the Colorado River Basin threaten regional energ…
🏢Big TechYouTube Updates View Metrics to Include TV Co-Viewing Estimates
YouTube is incorporating estimated co-viewing data into its public view counts to better reflect group consumption on television sets.
Apple Services Experience Widespread Outage
Apple users are reporting significant disruptions across iCloud, the App Store, and other core services, despite initial discrepancies in the company's status reporting.
⚖️Regulation & PolicyOpenAI Partners with Chilean Ministry to Advance Public Sector AI
OpenAI has signed a non-binding agreement with Chile's science ministry to integrate artificial intelligence into public administration and support local startups.