Hallucination
A hallucination is not a malfunction. It is the system doing exactly what it was built to do — produce plausible text — in a situation where plausible is not enough.
Why it matters
Hallucinations are dangerous specifically because they are fluent. A wrong answer that sounds uncertain gets checked; a wrong answer delivered in confident, well-formed prose gets acted on, which is where the real cost is.
- Invented policies, prices, or features stated as fact
- Citations to pages that do not exist
- Confident answers about things the system was never told
Why it happens
A language model predicts likely continuations of text. It has no separate store of facts to check against and no internal signal distinguishing 'I know this' from 'this is the kind of thing that usually follows'. Asked about your refund policy, producing a plausible refund policy is the model working correctly.
What actually reduces it
Grounding the answer in retrieved content removes the main cause. Explicit refusal behaviour handles the remainder by making 'I do not know' an acceptable output. Citations handle what is left by making verification cheap.
Grounding
Answer from retrieved passages, not from memory.
Refusal
Make declining an acceptable and expected outcome.
Citation
Attach the source so any claim can be checked in a click.
The counterintuitive part
A system that refuses more often is usually more trustworthy, not less capable. Optimising purely for answer rate selects for exactly the behaviour you are trying to eliminate.
Common questions
- Can hallucination be eliminated entirely?
- Not with certainty by any system that generates language. It can be made rare through grounding and refusal, and cheap to catch through citations. Treat any claim of complete elimination with suspicion.
- Is a wrong retrieval a hallucination?
- Strictly, no — that is a retrieval failure, where the model faithfully answered from the wrong passage. The distinction matters when debugging: check the citation to tell which one you are looking at.
From the blog
All posts- EvaluatingThe demo is a bad predictor of AI support qualityA polished demo tells you the product can answer questions it was prepared for. The useful evaluation happens at the edges — outside the script, on your content.Read
- EvaluatingWhat a citation should prove (and what most of them don’t)Citations look like grounding. Many are just related links. Here is the two-check test — resolves, and contains the claim — and what fails it.Read
- EvaluatingParaphrase testing: the five-minute retrieval checkA five-minute paraphrase test separates keyword matching from real retrieval. Same fact, three phrasings, one expected answer — including on Matter Chat.Read
Answer honestly. Capture the rest.
Point Matter Chat at your site and see what it can — and can't — answer. It's honest about both.
No credit card. 2 minute setup.
Every answer cites the source it came from. When there isn't one, it says so — and hands the visitor to you.
Installs on the tools you already run.