Large language model (LLM)
A large language model predicts what text plausibly comes next. Everything it does well and everything it does badly follows from that.
Why it matters
Most confusion about AI support comes from expecting a database and getting a text predictor. It does not look anything up unless you build retrieval around it, and it has no internal sense of which of its outputs are true.
- Expecting a model to 'know' private or recent facts
- Surprise that a confident answer was fabricated
- Assuming more training data would fix specific factual errors
What it actually does
Trained on large quantities of text, it learns statistical relationships between tokens and generates continuations one token at a time. Fluency emerges from that; factual reliability does not, because nothing in the process distinguishes true statements from likely-sounding ones.
Why retrieval is added
Because a model cannot know your content, useful products supply it at question time. That is the whole reason RAG exists — it is the bridge between a capable text generator and facts that are specific, private, and current.
Common questions
- Does an LLM know facts?
- It encodes statistical regularities that often produce correct facts, but it has no separate fact store and no reliable internal signal for what it does and does not know. That is precisely why grounding and citations matter.
- Would a bigger model fix accuracy?
- Not for facts about your business, which appear nowhere in any training set. Scale improves fluency and reasoning; retrieval is what supplies your specifics.
From the blog
All posts- MeasuringDeflection rate is the most overstated number in support automationCounting bot conversations as deflected tickets overstates the result. The honest version is a before-and-after on ticket volume, read next to satisfaction.Read
- EvaluatingHow to test an AI support tool before you trust itEvery AI support tool demos well, because demos ask questions the content covers. Four questions that separate them, and what a good answer looks like.Read
- MeasuringResolution rate vs deflection: stop mixing the twoResolution and deflection answer different questions. Mixing them inflates the result and hides whether customers actually got what they needed.Read
Answer honestly. Capture the rest.
Point Matter Chat at your site and see what it can — and can't — answer. It's honest about both.
No credit card. 2 minute setup.
Every answer cites the source it came from. When there isn't one, it says so — and hands the visitor to you.
Installs on the tools you already run.