AWS Builder Center
Amazon Bedrock Prompt Caching: What It Actually Solves, and the Details That Trip People Up

Amazon Bedrock Prompt Caching: What It Actually Solves, and the Details That Trip People Up

What Amazon Bedrock prompt caching actually does, the constraints that decide whether it works, and where it's not worth using.

If you're running any LLM application where the same context gets sent over and over (a system prompt, a big chunk of reference documentation, a tool schema, a codebase snippet), you're paying to reprocess that content on every single call. Prompt caching on Amazon Bedrock exists specifically to stop that waste, and it's one of the highest-leverage, lowest-effort optimizations available on Bedrock today.
This isn't a "top 10 tips" list. It's a walkthrough of how the feature actually works, where the real constraints are, and where I'd push back on treating it as a magic switch.

The problem in plain terms

Think about a document Q&A tool. A user uploads a 10-page PDF and asks five follow-up questions. Without caching, the model reads that entire 10-page document from scratch on every single question, five times, for content that didn't change at all. You're paying full price for redundant work every time, and the user is waiting for it too.
This pattern shows up everywhere: coding assistants that keep the same file open across many requests, customer support bots reusing a long system prompt, RAG pipelines injecting the same retrieved context into successive turns. Anywhere the same prefix repeats, prompt caching applies.
AWS announced this in preview at re:Invent 2024 and moved it to general availability in April 2025. The pitch is aggressive: up to 90% lower cost and up to 85% lower latency on the cached portion of a request. Those are ceiling numbers for large, highly-repeated prefixes, not what every workload will see, but the mechanism behind them is straightforward once you understand it.

How it actually works

Caching in Bedrock isn't about storing your prompt text somewhere. It's about storing the model's internal computed state for a segment of your prompt, so the model can skip reprocessing that segment on a later call.
You mark where the cacheable portion ends by inserting a checkpoint: a  cachePoint  block if you're using the Converse API, or a  cache_control  marker if you're using InvokeModel. Everything before that checkpoint is treated as the cacheable prefix. On your first call, Bedrock computes and stores that state (a "cache write"). On subsequent calls with the same prefix, Bedrock reuses the stored state instead of recomputing it (a "cache read"), and you pay a much lower rate for those tokens.
The catch that isn't obvious from the marketing copy: the prefix has to be identical, token for token, to hit the cache. If you insert a timestamp, reorder a JSON tool schema, or change even a small detail before the checkpoint, that's a cache miss and you pay full price again, as if caching wasn't there at all.

The constraints that actually matter

These are the details that decide whether caching works for your use case or quietly does nothing:
Minimum token thresholds. You can't cache tiny prompts. There's a per-model floor before a checkpoint is honored at all. For Claude 3.7 Sonnet on Bedrock, that's roughly 1,024 tokens; some other Claude and Nova models sit at similar or higher minimums, and this has shifted across model versions, so check the current figure for whichever model you're using. If your prefix is under the threshold, the request still succeeds. It just silently isn't cached. There's no error to warn you.
A short, resetting TTL. Bedrock's cache entries expire 5 minutes after last use by default, and that timer resets on every cache hit ("sliding window"). Some newer Claude models on Bedrock now also support a 1-hour TTL as an option. This design assumes bursty, back-and-forth usage: a chat session, a flurry of questions about one document. If your requests are spaced further apart than the TTL, you'll rebuild the cache every time and see none of the savings.
A cap on checkpoints per request. Most models allow up to 4 cache checkpoints in a single request, which matters if you're layering static system instructions, static tool definitions, and a static reference document separately.
Caching is regional. If you're using cross-region inference profiles, Bedrock may route two "identical" requests to different regions, and you'll get a cache miss even though nothing in your prompt changed. Worth knowing before you assume something's broken.
Not every model supports it. As of general availability, caching works on Claude 3.7 Sonnet, Claude 3.5 Haiku, newer Claude 4.x models, and the Amazon Nova family (Micro, Lite, Pro, Premier). Older or non-participating models will simply ignore cache markers.

What a request actually looks like

Using the Converse API, the shape is simple: you're adding one field, not restructuring your whole pipeline:
import boto3

client = boto3.client("bedrock-runtime", region_name="us-east-1")

response = client.converse(
modelId="anthropic.claude-3-7-sonnet-20250219-v1:0",
system=[
{"text": LONG_STATIC_SYSTEM_PROMPT},
{"cachePoint": {"type": "default"}}
],
messages=[
{"role": "user", "content": [{"text": user_question}]}
]
)

usage = response["usage"]
print(usage.get("cacheReadInputTokens"), usage.get("cacheWriteInputTokens"))

The response's  usage  object tells you what actually happened:  cacheWriteInputTokens  is nonzero on a cache miss (first write), and  cacheReadInputTokens  is nonzero on a cache hit. This is the field to watch in production. If  cacheReadInputTokens  stays at zero across requests you expected to hit the cache, something upstream is invalidating your prefix (often something silly, like a tool list that gets re-serialized in a different order between deployments).

Where it's genuinely worth using, and where it isn't

Good fit:
  • Document Q&A where users ask several questions about the same source in a short window
  • Coding assistants holding the same file(s) in context across a session
  • Chat agents with a long, unchanging system prompt and short user turns
  • Agents with large, static tool schemas attached to every call
Weak fit:
  • Low-traffic endpoints where requests are minutes apart, you'll be under the TTL constantly and mostly pay for cache writes with few reads
  • Prompts that are already small (under the per-model minimum), there's nothing to save
  • Highly dynamic prompts where the "static" part keeps changing in small ways (this defeats the whole mechanism, since even a one-token change before the checkpoint invalidates it)

The practical takeaway

Prompt caching is not a "flip it on and forget it" feature. It's a mechanism with real edges: token minimums, a short TTL, regional routing, and a strict requirement that the cached prefix be byte-for-byte identical between calls. Structure your prompts so the static, reusable part comes first, followed by the checkpoint. Then watch  cacheReadInputTokens  in production, not just your latency dashboard, because that's the number that tells you whether the cache is actually doing anything or silently missing every time.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article