Context windows: capacity is not attention

One budget, four claimants

The window is shared. A system instruction, whatever retrieval placed in front of the request, the prior turns of a conversation and the space the response itself will occupy all draw on the same total. The last of those is the one teams forget: if the input fills the window, there is nothing left for the answer, and the failure appears as an output that stops mid-sentence rather than as an error. Reserve output space explicitly and treat the remainder as the real input budget.

Position matters, and it is measurable

Material near the start and end of a long input is generally used more reliably than material buried in the middle. Rather than accepting or dismissing that as folklore, measure it on your own task: take a fact your prompt depends on, place it at the beginning, the middle and the end of an otherwise identical long input, and score the outputs. The result is specific to the candidate and the input length, and it is the single most useful thing you can learn about long-context behaviour in an afternoon.

Longer inputs are not free

Filling a window costs on three axes at once. It costs money, because input is charged. It costs time, because the input must be processed before the first output token appears. And it often costs accuracy, because more retrieved material means more nearly-relevant text competing with the passage that actually answers the question. A retrieval step that returns fewer, better passages frequently beats one that returns many and relies on the window to sort it out.

How to compare candidates on context

Compare at the length you will actually send, not at the maximum. Build inputs at your realistic median and worst case, keep them byte-identical across candidates, and record accuracy, time-to-first-token and cost together. A candidate with the smaller stated limit can be the better choice at your real length, and the stated limits will never tell you that.