Every frontier model shipped in 2026 advertises a context window of a million tokens or more. It’s the headline number on every launch post, and it’s the number people use to decide whether a document will fit.
It’s also the number most likely to mislead you. A model can accept a million tokens and still be measurably unreliable at 150,000. The gap between what fits and what the model can actually use is the single most under-discussed thing about working with LLMs right now — and unlike most claims in this space, there’s real measurement behind it.
What a context window actually contains
A context window is the total number of tokens a model can work with in one request. The part people miss is what “total” covers. Everything shares one budget:
- The system prompt
- The full conversation history, replayed on every turn
- Any documents, code, or search results you’ve attached
- The model’s own response
That last one catches people out. If you fill a 200K window with 199K tokens of input, you haven’t left room for an answer. Providers cap output separately too — Claude Fable 5, for instance, tops out at 128K output tokens per request regardless of how much window is free.
This is also why a long chat gets more expensive per message rather than staying flat. Every turn resends the accumulated history, so token 50 of a conversation is cheap and token 50,000 is not.
The 2026 numbers
| Model | Context window | Notes |
|---|---|---|
| Claude Fable 5 (Anthropic) | 1M | Up to 128K output per request |
| Claude Opus 5 (Anthropic) | 1M | |
| Claude Sonnet 5 (Anthropic) | 1M | |
| GPT-5.5 (OpenAI) | 1M | Input priced 2x, output 1.5x beyond 272K |
| GPT-5.4 (OpenAI) | 1.1M | |
| Gemini 3.1 Pro (Google) | 1M | Higher token rate past 200K |
| Gemini 3.6 Flash (Google) | 1M |
Two of these charge you more for using the top of their own window. GPT-5.5 doubles input pricing past 272K tokens; Gemini 3.1 Pro moves to a higher rate past 200K. The window is a million tokens, but it isn’t a million tokens at one price — worth knowing before you architect around “it all fits.”
What happens when you go over
It depends where you are.
In a chat interface, you usually won’t see an error. The app quietly drops or summarizes older messages to make room. This is why a long conversation sometimes seems to forget something you said an hour ago — it isn’t a memory bug, the message is simply no longer in the window.
Through the API, you get a hard error. The request is rejected before inference starts, so you’re not billed for a truncated attempt, but you do have to handle it. If your app builds prompts from user-supplied documents, this is the failure mode you’ll hit in production first.
Neither behavior is the interesting problem, though. The interesting problem happens before you reach the limit.
Context rot: the part the spec sheet doesn’t tell you
In 2025, Chroma published research testing 18 frontier models — including GPT-4.1, the Claude 4 family, Gemini 2.5, and Qwen3 — on how performance changes as input length grows. The finding was consistent across every model tested: accuracy degrades as context fills, well before the advertised limit. On retrieval tasks, models dropped 20–50% going from 10K to 100K tokens of input. The term that stuck for this is context rot.
Three findings from that work are worth internalizing, because they change how you should structure prompts:
Position matters, and the middle is worst. Performance follows a U-shaped curve: information at the very beginning or the very end of the context is recovered reliably, while information buried in the middle can degrade by more than 30%. If something is critical, it does not belong in the middle of a long document dump.
Semantic similarity hurts more than raw length. When the thing you’re looking for is obviously distinct from everything around it, models find it fine even in long contexts. When the surrounding text contains passages that look like plausible answers, accuracy drops sharply — and the drop compounds as length grows. A 100K-token context of unrelated material is easier for the model than a 50K-token context full of near-misses.
Coherent documents can perform worse than shuffled ones. This is the counterintuitive one. In Chroma’s tests, models did better when the surrounding text had its logical flow broken up. Structural coherence appears to give the model more plausible-but-wrong threads to follow.
Taken together, the practical rule is that effective context is meaningfully smaller than advertised context. A useful working assumption is that reliability starts degrading somewhere around a third of the way into the stated window, and that “it fit, so it worked” is not a safe inference.
What to do about it
Don’t fill the window just because you can. The instinct to paste everything and let the model sort it out is exactly what context rot punishes. Retrieving the relevant 20K tokens beats dumping 500K.
Put critical instructions at the start or the end. Given the U-shaped curve, the middle of a long prompt is the worst place for anything that must not be missed. Repeating key constraints at the end of a long document is cheap insurance.
Use retrieval and long context together. The 2026 default architecture isn’t “RAG or long context” — it’s retrieve a relevant 50K–200K, then reason over that with a long-context model. You get the precision of retrieval and the reasoning quality of a large window, without paying the rot tax on either.
Compact conversation history instead of replaying it. Summarize old turns rather than resending them verbatim. This cuts cost and reduces the distractor material that degrades retrieval.
Measure before you assume. Before shipping something that relies on a 400K-token prompt, test it at that length with realistic distractors — not with a clean needle in clean hay. The benchmark that matters is your data.
FAQ
Is a bigger context window always better?
No. A larger window gives you more room, but accuracy per token doesn’t hold constant as you fill it. A well-retrieved 30K-token prompt frequently outperforms a 500K-token dump on the same question, and costs far less.
How many pages fit in a 1M-token context window?
Roughly 1,500–3,000 pages of plain prose, depending on density and language. But “fits” and “the model will reliably use all of it” are different claims — see context rot above.
Does the context window reset between messages?
In a chat interface, no — the history accumulates and is resent each turn, which is why long chats eventually hit the limit. Through the API, each request is independent: you control exactly what goes in the window every time.
Why did the model forget something I said earlier?
Almost always because it left the window. Chat interfaces drop or summarize old turns silently to make room, so the message isn’t in the model’s input anymore, no matter how clearly you said it.
Do input and output share the same limit?
Yes, they draw on the same budget, and providers additionally cap output separately. Leave room for the answer — a nearly-full window can’t produce a long response.
The practical first step is knowing how big your prompt actually is. Our free token counter gives you the number instantly, entirely in your browser. And if you’re fuzzy on what a token is in the first place, start with what LLM tokens are and how they’re counted.
Related Tools:
- Token Counter - Measure prompt size before you hit the limit