count_tokens gave a number that didn't match the bill

The mismatch that looks like a billing bug

A team runs the same text through the counting endpoint before sending a real request, gets a number back, sends the request, and the actual bill doesn't match what was counted — sometimes by a wide margin. The counting call didn't fail, and nothing about it looked wrong; it just counted against the wrong model.

Why the model id you count against actually matters

Token count for identical text is not a fixed property of the text alone — it depends on which model's tokenizer is doing the counting, and different model generations in the current lineup genuinely tokenize the same input differently. Counting against one model id and then sending the real request to a different one produces a count that was accurate for a model you never actually called, which is functionally the same as not having counted at all, just with more false confidence attached to the wrong number.

How the mismatch usually sneaks in

This rarely happens because someone deliberately counted against the wrong model — it happens because the model id used for counting and the model id used for the real request live in different parts of the code, or get set at different times, and drift apart silently. A model-picker function that decides the real request's model dynamically, paired with a counting call that has a model id hardcoded from an earlier version of the code, is the classic setup for this: the request-time model changes, and the counting call never gets updated to match.

The fix

Derive the model id passed to the counting call from the exact same variable that decides which model the real request goes to — never from a separate hardcoded value, even one that was correct when it was written. If your pipeline routes across multiple models dynamically, count immediately before each send using that request's actual target model, not a single count reused across different destinations. The token & cost estimator shows the spread across every current model side by side specifically so this kind of mismatch is visible before it reaches production.