Token
A token is roughly three-quarters of a word. It is what models process and what usage is measured in.
Why it matters
Tokens are the unit models charge in. Matter Chat prices and caps in replies instead, so you never set a token number yourself — but your usage chart counts them, and an intuition for the unit turns an abstract figure into one you can reason about.
- Usage limits expressed in units nobody has a feel for
- No sense of what a normal conversation actually consumes
- Cost caps set to arbitrary round numbers
What counts as a token
Text is split into pieces that are often but not always whole words — common words are usually one token, longer or unusual ones are several. English averages roughly four characters per token, so a hundred words is about 130 tokens.
Where they are consumed
Both directions count, and in a retrieval system the input is usually the larger share: the question, the retrieved passages, and the instructions all enter the context, and the answer is generated on top.
Input
Question, retrieved passages, and system instructions.
Output
The generated answer.
Common questions
- How many tokens is a typical support conversation?
- It varies with how much content is retrieved, but the retrieved passages usually dominate rather than the visible exchange. The practical approach is to watch a fortnight of real usage in your analytics, then set your daily reply cap above your busiest day.
- Does Matter Chat cap tokens or replies?
- Replies. Tokens are what a reply costs underneath, and the analytics chart them, but the cap you set is a number of answers per day. That is the unit you are billed in and the one you can reason about — a token cap is precise about cost and useless for predicting how many customers you can serve before the bot goes quiet.
From the blog
All posts- MeasuringDeflection rate is the most overstated number in support automationCounting bot conversations as deflected tickets overstates the result. The honest version is a before-and-after on ticket volume, read next to satisfaction.Read
- EvaluatingHow to test an AI support tool before you trust itEvery AI support tool demos well, because demos ask questions the content covers. Four questions that separate them, and what a good answer looks like.Read
- MeasuringResolution rate vs deflection: stop mixing the twoResolution and deflection answer different questions. Mixing them inflates the result and hides whether customers actually got what they needed.Read
Answer honestly. Capture the rest.
Point Matter Chat at your site and see what it can — and can't — answer. It's honest about both.
No credit card. 2 minute setup.
Every answer cites the source it came from. When there isn't one, it says so — and hands the visitor to you.
Installs on the tools you already run.