Why your prompt caches on one model and not another

The assumption that quietly breaks

A prompt engineered and tested against one model, where caching works reliably, gets pointed at a different model — for cost, a fallback path, or an upgrade — and caching silently stops working. Nothing errors. The request succeeds. It just never caches, and the only reason this counts as "why your prompt caches on one model and not another" rather than a generic caching bug is that the cause is specifically the model, not the prompt or the request logic.

The minimum cacheable length is not a fixed constant

The most common cause is straightforward once you know to check it: the minimum prefix length a model requires before anything can cache at all is not the same figure across the current lineup, and it doesn't move in the direction you'd naturally assume — newer isn't uniformly lower. A prefix that comfortably clears one model's minimum can fall short of another's, and a request routed dynamically across models, or a fallback chain, can hit this exact gap without anyone having changed the prompt at all. See the minimum-cacheable-prefix table for the full picture — it's genuinely not monotonic, which is the whole reason this happens.

Caches don't carry across models even above the minimum

Even where both models clear their respective minimums, a cache entry is scoped to the specific model that created it — switching models mid-conversation, or routing the same logical request to a different model on a retry, starts a fresh cache with nothing to read from, regardless of how long the prefix is or how far above minimum it sits.

Fallback and retry logic is where this hides best

The riskiest version of this isn't a deliberate model swap — it's a fallback chain that silently routes to a different model during a retry or an availability issue, without anyone treating that as a caching-relevant event. A system that appeared to cache reliably in testing, because testing never exercised the fallback path, can behave completely differently the first time the fallback actually triggers in production, exactly when nobody's watching closely because something else already went wrong to trigger the fallback in the first place.

Building a caching setup that tolerates model variation

If your system routes across more than one model — by design or as a fallback — pad your cache-worthy prefixes above the highest minimum any model in your routing set requires, not just the lowest one your primary model needs. That single change makes caching resilient to which model actually ends up serving a given request, rather than working only when the primary path is taken.

A/B testing infrastructure or an evaluation harness that cycles a prompt across several models to compare quality often isn't built with caching economics in mind at all — each variant run against a different model starts its own separate cache, and a harness that runs frequently can rack up repeated cache-write costs across every model variant it tests, on top of whatever the actual comparison was trying to measure. If your eval infrastructure runs the same prompt across multiple models regularly, budget for the caching cost of that multiplicity explicitly rather than assuming it's a rounding error.

The check that catches this before it reaches production

Before trusting a caching setup that spans more than one model, deliberately send a test request to every model in your routing set and confirm each one shows the expected cache behaviour individually — don't infer coverage from testing against your primary model alone and assuming a fallback or secondary model behaves the same way just because the request looked identical.

See writing system prompts that cache reliably for the prompt-construction half of this problem, and cache_read_input_tokens is 0 with no error for the fuller debugging checklist beyond the model-specific causes covered here.