What a 1M-token context window actually buys you

A capability that arrived faster than the habits to use it well

A context window large enough to hold a genuinely large codebase or document set is still new enough, relative to how software teams have worked for most of their careers, that the habits around using it well haven't caught up to the capability itself. That gap is worth naming directly: the number on a spec sheet changed quickly; how people actually decide what to send changed more slowly, and a lot of teams are currently running last generation's scoping habits — or no scoping habits at all — against this generation's ceiling.

The engineering culture this capability inherited, and didn't fully replace

For years, the discipline around retrieval and summarisation existed because it had to — the alternative was a request that simply didn't fit. That constraint, annoying as it was, did something useful almost as a side effect: it forced a genuine decision about what mattered before anything got sent. A large window removes the forcing function without removing the underlying question, and the teams handling this transition best are the ones who kept asking the question on purpose after the constraint that used to ask it for them went away.

The "might as well" reflex, and why it's more expensive than it feels

There's a specific, very human reflex worth naming: if something technically fits, and you're not sure exactly how much of it you need, the path of least resistance is sending all of it — "might as well, it fits." That reflex costs nothing to indulge in a demo or a one-off exploration. It costs real, compounding money the moment the same pattern runs as a repeated, automated request across real traffic, because "might as well" scoping decisions don't get revisited once they're wired into a pipeline; they just run, unquestioned, at whatever volume the system sees.

Why breadth is a real value, not just a cost to be minimised

None of this is an argument for scoping context down reflexively either. Some tasks are genuinely better served by broad, simultaneous visibility — the value comes specifically from being able to see connections across a large body of material at once, not from any single narrow slice of it. The point isn't "smaller is always better"; it's that breadth should be a deliberate answer to "does this task's value scale with how much it can see at once," not a default applied uniformly regardless of what a given request actually needs to do its job.

What actually distinguishes teams that use this well

The distinguishing factor isn't sophistication with the underlying technology — it's whether someone, at some point, is still asking the scoping question a tight constraint used to ask automatically. A team that never lost that habit treats a large window as expanded room to make a better-informed decision. A team that let the habit lapse treats it as a reason not to decide at all, and pays the difference in every request that sends more than the task in front of it actually needed.

What this looks like on a team that's actually getting it right

Concretely, it doesn't look like a rule ("never send more than X") or a blanket policy ("always scope aggressively") — it looks like a question that gets asked at the point a new request pattern is designed, every time, rather than answered once and assumed to still hold. A team that revisits the question as workloads evolve catches the case where a pattern that used to need broad context no longer does, and the case where a pattern that used to be narrow has grown into something that genuinely benefits from more. A rule fixed in place at design time can't track either kind of drift; a live habit of asking can.

Why this is a cost story and a quality story at the same time

The financial angle is the easiest one to measure, but it isn't the only one worth caring about. An unscoped request doesn't just cost more — a response built from too much undifferentiated context can genuinely be worse, harder for a reader to trust, than one built from exactly what the task needed and nothing else. Treating scoping purely as a cost-control exercise undersells it; done well, it's also doing real work toward a better answer, not just a cheaper one.

See budgeting for a 1M-token context window for the practical mechanics of scoping, caching, and reserved-budget decisions this essay deliberately doesn't re-walk, and the context-window planner for sizing your own actual content directly rather than reasoning about it in the abstract.