A working prompt-caching implementation guide

Start from what stays stable, not from where it's convenient to add a marker

The instinct when implementing caching is to mark a breakpoint wherever it's syntactically convenient — after the system prompt, say, because that's the obvious first block. The better starting point is asking what in your actual request genuinely stays byte-identical across many calls, and marking breakpoints around that content specifically, even if it means splitting a request in a less obvious place. A breakpoint placed after content that only sometimes repeats caches inconsistently, which is a worse outcome than not caching at all, because it hides an intermittent problem behind occasional cache hits that make it look like it's working.

Order content from most stable to least stable

Everything before a cache breakpoint has to be byte-identical to a prior request for that breakpoint to hit, which means content order matters as much as content itself. Put your most stable material first — static instructions, shared examples, anything that essentially never changes — and your most volatile material last: the specific user message, anything with a timestamp or session-specific value. A request that interleaves stable and volatile content throughout, rather than front-loading what's stable, forces you to either cache very little or place breakpoints awkwardly around scattered stable fragments.

The tool list is part of the cacheable prefix too

Tool definitions render at the very start of a request, ahead of the system prompt, which means they're part of whatever prefix you're trying to cache whether you think of them that way or not. Building the tool list dynamically — per user, per mode, per feature flag — changes that prefix on every request even if your system prompt and instructions are otherwise completely static, and invalidates caching for the whole request, not just for the tools themselves. If your tool list needs to vary, that variation belongs as late in the request as your tool-use requirements allow, not baked into the earliest, otherwise-stable part of the prompt.

Choosing between the two TTLs

The shorter TTL suits a prefix that gets reused in quick, dense bursts — a batch of similar requests fired close together — where the cache is unlikely to still be useful much later anyway. The longer TTL suits a prefix reused more sporadically but still reliably within an hour, where a shorter window would let the cache expire between genuinely-related requests that just happen to be spaced further apart. Pick based on your actual request cadence, not on whichever default your SDK ships with — the wrong TTL choice for your pattern either expires a cache you needed or pays for a longer write than your reuse pattern justifies.

Testing caching locally before it matters in production

A caching bug is much cheaper to catch in a local test than in a production bill three weeks later, and it's straightforward to test for directly: send the same request twice in immediate succession and check that the second one shows a cache read rather than a fresh charge. Build this into your test suite as an explicit assertion rather than a manual spot-check, so a future change that accidentally introduces dynamic content into what was a stable prefix gets caught by CI rather than by someone eventually noticing the bill looks larger than expected.

Verifying it's actually working, not just configured

A caching setup that's correctly configured in code can still fail to cache in practice for reasons that produce no error at all — a prefix below the minimum length a specific model requires, a piece of dynamic content earlier in the prompt than intended, a model switch mid-session. Don't treat "no errors" as confirmation that caching is working; check the actual usage block against what you expected on a sample of real requests before trusting the implementation at scale. See cache_read_input_tokens is 0 with no error for the full list of causes worth checking against your own setup, and the prompt-caching savings calculator to confirm your request pattern is actually one that benefits from caching before you build the implementation around it.