Launch day pressure pushes teams to ship the widget and “iterate in production”. That works for button colours. It is a bad plan for a system that speaks in your brand’s voice about money, deliveries, and policies. Twenty minutes of staged checks prevents a month of apology emails.
These five questions are the building-side twin of the evaluation tests we publish for vendor shopping. Run them on your own bot. Failures point at sources, settings, or copy — rarely at “needing a smarter model”.
1. What hosts are we answering from?
Open the knowledge source list. Confirm every indexed URL is a production host you intend to cite. No staging, no leftover preview deploys, no forgotten demo site. If you see a non-production domain, stop — see staging domains will poison your answers before anything else.
2. Do citations prove the claim?
Ask five things you already know the answers to — refund window, price, hours, a core how-to. Open each citation. The link must resolve, and the page must contain the claim. A homepage citation is decoration. Fix retrieval or page structure before launch; the first crawl guide covers the review loop.
3. What happens outside the content?
Ask something plausible that you do not publish. Acceptable: a plain refusal and a path to a person or form. Unacceptable: a fluent invention. Also ask the categories you banned in the persona. If those fire incorrectly, fix persona rules and refusal design now.
4. Can a human take over with context?
Type that you want to speak to someone. Count turns — one is correct. Complete a test escalation and confirm the transcript arrives where the team actually works, including outside hours behaviour. Details in handle escalations.
5. What stops this from spending without a ceiling?
Confirm domain locks include every real origin (www and apex are separate), per-visitor rate limits are on, and a daily cap sits above a normal day but at a number you would accept on a bad day. The public key is visible by design; domain locks and spend caps are what make that safe.
| Question | Pass | Fail |
|---|---|---|
| Hosts | Production-only sources | Staging or preview in the index |
| Citations | Link contains the claim | Homepage or unrelated page |
| Outside content | Refusal + next step | Fluent invention |
| Human | One-turn handoff + transcript | Loop or empty ticket |
| Ceiling | Locks + rate limit + daily cap | Open key on any origin |
Optional sixth: languages you sell in
If you expect non-English traffic, repeat the outside-content refusal test in those languages even when sources are English-only. Fabrication is hardest to catch in a language nobody on the team reads. See multilingual setup.
Ship only when the dull checks pass
A pretty launcher icon does not compensate for a poisoned index or a missing handoff. When all five pass on a staging page with production knowledge and locked domains, you are ready for real traffic — then watch refusals and citations for the first fortnight rather than staring at answer rate alone.
