Retrieval-augmented generation (RAG)
Retrieval-augmented generation means looking up the answer before writing it, rather than recalling it from training.
Why it matters
A language model on its own answers from statistical patterns learned in training. It has never read your refund policy, and if asked about it will produce something plausible. RAG is the architecture that fixes this by fetching your actual content first.
- Models cannot know anything private, recent, or specific to you
- Retraining a model on your content is slow, costly, and stale on arrival
- Answers without a source cannot be verified by the person reading them
How it works
Your content is split into passages and indexed. When a question arrives, the system searches that index for the passages most likely to contain the answer, puts them into the model's context, and asks it to answer using only that material. The passages it used become the citation.
Retrieve
Find passages semantically related to the question.
Augment
Place them in the model's context as the material to use.
Generate
Produce an answer constrained to that material.
Why it beats fine-tuning for this job
Fine-tuning adjusts a model's behaviour, not its facts, and any facts it does absorb are frozen at training time. RAG updates the moment your content does — publish a page, re-index, and the answer changes. It also produces a citation, which fine-tuning cannot.
What it does not solve
RAG makes the right material available; it does not guarantee the model uses it well, and it cannot retrieve what was never indexed. Retrieval quality — chunking, search, ranking — is where most real-world accuracy problems actually live.
Common questions
- Does RAG stop hallucination completely?
- No. It removes the most common cause — answering from memory about things it was never told — and makes the rest verifiable by attaching a source. Anything that generates language can still err, which is why the citation matters as much as the retrieval.
- Is RAG the same as search?
- It contains search but does not stop there. Search returns documents for you to read; RAG uses those documents to compose a direct answer, then shows you where it came from.
Related terms
From the blog
All posts- EvaluatingParaphrase testing: the five-minute retrieval checkA five-minute paraphrase test separates keyword matching from real retrieval. Same fact, three phrasings, one expected answer — including on Matter Chat.Read
- BuildingGrounding without the mystiqueStrip the jargon and grounding is simple — supply the passages, require the model to use them, refuse when they are missing. Here is how to check it is actually happening.Read
- BuildingChunking mistakes that look like model failuresSplit a policy table from its header, bury the exception in a footer, or index a nav-heavy page as one blob — and the model will look broken while retrieval did the damage.Read
Answer honestly. Capture the rest.
Point Matter Chat at your site and see what it can — and can't — answer. It's honest about both.
No credit card. 2 minute setup.
Every answer cites the source it came from. When there isn't one, it says so — and hands the visitor to you.
Installs on the tools you already run.