Chunking
Chunking is the least discussed and most consequential decision in a retrieval system.
Why it matters
Retrieval returns chunks, not documents. A chunk too large dilutes its own meaning and wastes context; one too small loses the surrounding information that made it intelligible. Both produce answers that feel subtly wrong.
- Answers citing the right page but missing the relevant detail
- Retrieved passages that make no sense without their heading
- Long pages that retrieve for everything and answer nothing precisely
The trade-off
Larger chunks carry more context and match less precisely, because a single vector now represents several ideas. Smaller chunks match precisely and may omit what made them meaningful — a paragraph that says 'this does not apply to annual plans' is dangerous without the paragraph above it.
Why structure helps
Splitting on headings and paragraph boundaries produces chunks that correspond to actual ideas, which is why well-structured documentation retrieves better than the same content written as one continuous essay. Your heading structure is doing real work here.
Common questions
- Do I need to configure chunking?
- Not in a managed product. What you control is the structure of your content, and clear headings with one topic per section improve retrieval more than any parameter would.
- Why does my long page answer questions badly?
- Most likely it covers several topics, so it retrieves for all of them and the matched passage is often about a different section than the question. Splitting it usually fixes it.
From the blog
All posts- BuildingGrounding without the mystiqueStrip the jargon and grounding is simple — supply the passages, require the model to use them, refuse when they are missing. Here is how to check it is actually happening.Read
- BuildingChunking mistakes that look like model failuresSplit a policy table from its header, bury the exception in a footer, or index a nav-heavy page as one blob — and the model will look broken while retrieval did the damage.Read
- MeasuringDeflection rate is the most overstated number in support automationCounting bot conversations as deflected tickets overstates the result. The honest version is a before-and-after on ticket volume, read next to satisfaction.Read
Answer honestly. Capture the rest.
Point Matter Chat at your site and see what it can — and can't — answer. It's honest about both.
No credit card. 2 minute setup.
Every answer cites the source it came from. When there isn't one, it says so — and hands the visitor to you.
Installs on the tools you already run.