One agent turn can lose your Bedrock prompt cache
AWS says Bedrock looks back about 20 blocks for a cached prompt. I measured it: 21 hit, 22 miss. Plus other ways the cache resets without an error.
Prompt caching is one of the biggest cost levers for Claude on Amazon Bedrock: cached prompt text is billed at about a tenth of the normal input price. It also fails without an error. The request still succeeds; you just pay full price again, plus the cache write premium.
The AWS docs give the key rule for agents only roughly: Bedrock "looks back approximately 20 content blocks" from a cache checkpoint to find an earlier entry. So I measured it on live Bedrock (Claude Sonnet 4.6, us-west-2, two runs per size with fresh prompts).
With 21 blocks added since the previous cache point, the next request read the cache. With 22, it missed, every time. Every tool call and every tool result is a block, so one agent turn with 11 parallel tool calls is enough to lose the cache. Anthropic's own API doesn't have this problem, because it counts a run of tool calls as one position; Bedrock counts each block.
The fix: keep the previous cache point when a turn adds more than about 18 blocks. Bedrock allows four cache points per request, so there is usually room.
Other ways the cache resets without an error
I tested these on live Bedrock too:
- Changing thinking or effort settings between calls. Switching thinking on, changing its budget, or using a different effort level made the next call read nothing from the cache, system prompt included. No effort setting behaves the same as "high".
- Switching tool choice between auto/none and any/tool. The tools and system prompt were still read, but the conversation part was written again.
- A cache point with nothing before it in its own list, for example at the start of a message. Bedrock rejects the request: "There is nothing available to cache."
- A cachePoint nested inside a toolResult's content in the Converse API. boto3 refuses to send it, and over plain HTTP Bedrock accepts the request but caches nothing.
- Anything dynamic before the cache point, such as today's date or a request ID in the system prompt.
Temperature and max tokens did not affect the cache.
Checking your own account
In CloudWatch, the AWS/Bedrock namespace has CacheReadInputTokenCount and CacheWriteInputTokenCount by model ID. In steady traffic, reads should be many times writes. Big writes with reads near zero means caching is on but missing, which costs more than not caching at all.
For the reason behind a miss, I built a small free tool, CacheCanary (https://github.com/Haarris/cachecanary ). It reads Bedrock invocation logs for the hit rate and what the misses cost, and compares two saved requests to tell you why the second one missed.
The full write-up, with the bug reports from LiteLLM, Cline, pydantic-ai, Strands and others: https://harisfarooq.substack.com/p/five-ways-claude-prompt-caching-quietly
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article