Every support bot sits on top of a website that is incomplete — not because the team is lazy, but because real businesses ship faster than documentation. A content gap is the measurable form of that incompleteness: a recurring visitor question your public material does not answer. Most teams discover gaps accidentally in tickets. Fewer count them as a metric with owners and targets.
When gaps stay informal, automation optimises around them badly — inventing answers, bloating persona prompts, or celebrating answer rate while trust leaks. When gaps become a metric, the bot’s refusals turn into a content sprint board finance can understand.
Without the metric, gaps become anecdote — one loud ticket drives a FAQ, twenty quiet refusals in the widget do not. Formalising the count fixes that imbalance.
Age open gaps in days. A gap open ninety days with a rising refusal count is a prioritisation failure rather than a content problem: someone owns the metric but not the work.
Define a gap precisely
A gap is not “the bot failed.” It is a specific mismatch:
- Question appeared at least N times in widget or tickets (pick N — three is a fine start).
- No public page contains the answer a reasonable customer expects.
- A page could plausibly host the answer — it is not purely account-specific.
Instrument gaps from two feeds
Refusals are the clean feed: honest declines where content is absent. Tickets are the noisy feed: humans answered, but the bot should have been able to. Merge them weekly into themed rows with shared IDs.
| Field | Why it matters |
|---|---|
| Theme label | Rolls up paraphrases |
| Weekly refusal count | Demand signal |
| Ticket overlap | Confirms business impact |
| Severity tag | Legal, revenue, annoyance |
| Owner | Someone’s sprint, not “the bot” |
| Status | Open, drafting, shipped, verified |
Gap closure rate: the metric executives need
Count open gaps at month start. Count gaps closed with a shipped page and verified recrawl. Gap closure rate = closed / (open + new). Pair with regression checks: after closure, refusals on that theme should fall and sampled answers should pass the citation claim test.
This metric improves when documentation ships — not when someone lowers refusal thresholds. If closure rate is flat while answer rate rises, you are covering gaps with guesses. That shows up later in wrong-answer audits, not in the gap metric unless you are honest about reclassifying inventions as incidents.
Cadence and accountability
- Weekly: rank top five open gaps from refusal list.
- Biweekly: assign writer, link draft ticket.
- On publish: trigger recrawl, run three paraphrase checks.
- Monthly: report open count, closure rate, oldest open gap age.
Include gap metrics in the monthly automation report. Finance grasps “we closed six of nine documented gaps” faster than model upgrades.
Set a target closure rate, not zero open gaps — zero is fantasy on a living site. A steady seventy percent closure on ranked gaps beats a heroic month followed by neglect.
When gaps reveal product, not copy
“Some gaps close with a paragraph. Some close with a feature. Metrics tell you which is which by watching refusals persist after good copy ships.”
If a theme stays open through multiple well-written pages, stop writing. Escalation design, tooling, or policy change owns it. Content gaps as a metric keeps automation tethered to the site you have, and shows when the site itself is the bottleneck. That is more valuable than any vanity containment chart.
Review closed gaps quarterly for regression — pages deleted, redirects broken, policy reversed. A closed gap that reopens is a recrawl alert, not a model problem.
