Context window
The context window is the model's working memory for a single request — and it is finite.
Why it matters
It explains why an assistant cannot simply be handed your entire documentation. Retrieval exists because the window is finite, and filling it indiscriminately makes answers worse rather than better.
- Expecting a model to read an entire site per question
- Assuming more supplied context always improves the answer
- Long conversations losing track of earlier detail
What it holds
Everything the model considers for one request: instructions, conversation history, retrieved passages, and the answer being generated. All of it competes for the same finite budget.
Why more is not better
Filling the window with loosely relevant passages measurably degrades answers — relevant detail gets diluted among noise. Selecting fewer, better passages beats supplying more, which is why retrieval quality matters more than window size.
Common questions
- Do larger context windows remove the need for retrieval?
- No. Even where a whole site would fit, filling the window with mostly irrelevant text degrades answer quality and costs far more. Selecting the right passages remains the job.
- Why does a long conversation lose earlier detail?
- Because history competes with retrieved content for the same finite window. Older turns are eventually dropped to make room for the material needed to answer the current question.
From the blog
All posts- BuildingEscalation that keeps the conversation contextHand over on refusal and on request, send the full transcript, and be honest about office hours. Context is the whole point of escalation.Read
- MeasuringDeflection rate is the most overstated number in support automationCounting bot conversations as deflected tickets overstates the result. The honest version is a before-and-after on ticket volume, read next to satisfaction.Read
- EvaluatingHow to test an AI support tool before you trust itEvery AI support tool demos well, because demos ask questions the content covers. Four questions that separate them, and what a good answer looks like.Read
Answer honestly. Capture the rest.
Point Matter Chat at your site and see what it can — and can't — answer. It's honest about both.
No credit card. 2 minute setup.
Every answer cites the source it came from. When there isn't one, it says so — and hands the visitor to you.
Installs on the tools you already run.