Support teams usually blame the model when an answer is almost right. The citation points at the shipping page. The tone is fine. The number is wrong by a day, or the exception is missing. That pattern is rarely a reasoning failure. It is usually chunking: the retrieved passage never contained the whole fact.
Chunking is the unglamorous step where your pages are split into passages before indexing. Get the splits wrong and retrieval will hand the model a fragment that looks relevant and is incomplete. The model then does what models do — it completes the thought. The completion looks like hallucination. The root cause was scissors.
The failure mode that fools everyone
Open the citation. If the page as a whole contains the correct policy, but the answer still missed a condition, ask a narrower question: did the retrieved chunk contain it? Tables are the classic trap. A header row in one chunk, rates in the next, footnotes in a third — and the assistant retrieves the rates without the “excludes remote areas” line.
Mistakes that keep showing up
- Splitting mid-list or mid-table so the retrieved piece has numbers without labels.
- Treating a long FAQ as one chunk, so retrieval scores the page high and the model skims the wrong Q&A pair.
- Indexing chrome — cookie banners, nav, footer promos — as if it were policy text.
- Separating “Returns” from “Exceptions” across templates so the common case retrieves cleanly and the edge case never does.
- Huge chunks that dilute the match: the right sentence is present but drowned by unrelated sections.
None of these require a different model. They require pages written as retrievable units: one job per section, with exceptions sitting next to the rule they modify. That is the same discipline as writing help pages an assistant can answer from.
How to tell chunking from hallucination
Use the citation as a fork. If the cited page does not support the claim at all, you are looking at ungrounded generation or a wrong-document retrieve. If the page supports the claim but only outside the likely passage boundaries (a footnote, a later H2, a sidebar), prefer a chunking or ranking fix before you touch prompts.
| What you find | Likely cause | Fix first |
|---|---|---|
| Claim absent from whole page | Hallucination or wrong source | Refusal + content gap |
| Claim on page, missing from answer | Incomplete chunk or weak rank | Restructure / re-chunk |
| Right section, stale number | Index lag after a rewrite | Recrawl |
| Related page, wrong product line | Ambiguous titles / thin chunks | Disambiguate pages |
Practical rules that survive contact with a real site
- Keep atomic facts together: the rule, the number, and the exception in one passage when you can.
- Prefer section-sized chunks over whole-page blobs and over sentence salad.
- Strip repeated nav and footer noise from what gets embedded.
- After a docs rewrite, re-crawl before you judge the model — stale chunks look like stubborn errors.
- When testing, ask for the exception on purpose; happy-path questions hide split failures.
What to change this week
Pull five wrong answers from the last month. For each, open the source and mark where the missing clause lives. If three or more are “right page, wrong fragment,” stop debating temperature and fix the pages: pull exceptions up, split mega-FAQs, give tables captions that travel with the rows.
Then re-index and ask the same five again. If the answers move, the model was never the problem. What you needed was passages that still make sense read alone, which is all a chunk ever is to the system.
