The Knowledge Gap Curve: How KB Quality Predicts Email Resolution Rate

Two subscription companies went live within a month of each other. Similar ticket volume, similar inbox mix, the same product on our side. At week twelve one was resolving 76% of emails autonomously and the other was at 48%.

The second team assumed they'd been given a worse model. They hadn't. They'd been given the same one, pointed at a knowledge base scoring 43 out of 100 instead of 78.

This is the most common conversation we have with customers, and for a long time we answered it with intuition. So we scored the knowledge base of all 42 deployments in our benchmark cohort at go-live and checked it against where each one landed at week twelve.

How we score a knowledge base

Counting articles tells you nothing, so we score four things that we can measure without reading every document by hand. Together they make a 0 to 100 score.

• Intent coverage (40 points): the share of the top 50 inbound intents that have at least one authoritative document behind them. Not a related document. One that answers the question.
• Answer specificity (25 points): the share of documents that state the actual decision rule, including thresholds, eligibility conditions, and timelines, rather than describing that a policy exists.
• Freshness (20 points): the share of articles reviewed or updated in the last 180 days.
• Contradiction rate (15 points): inverted. How many of the top 50 intents have two or more documents that give conflicting answers.

A knowledge base written for humans is a map with all the roads drawn and none of the distances marked. A person can work around that by asking a colleague. An agent can't.

The curve

Grouping the cohort by go-live KB score, against median autonomous resolution rate at week twelve:

• Below 40: 41% resolution
• 40 to 54: 52%
• 55 to 69: 64%
• 70 to 84: 74%
• 85 and above: 79%

The composite score correlated with week-12 resolution rate at r = 0.81, which works out to about two thirds of the variance between deployments explained by one input measured before launch. Nothing else we track comes close. Ticket volume, company size, industry, and even the number of connected integrations all correlate more weakly.

That's the sales-enabling version of this finding, and it's also the uncomfortable one, because knowledge quality is the variable a vendor has the least control over.

The number of articles doesn't matter

Article count correlated with resolution rate at r = 0.11. Statistically that's noise.

The clearest illustration in the dataset: a 60-article knowledge base scoring 82 outperformed a 900-article knowledge base scoring 44 by 26 points at week twelve. The large one had years of accumulated documentation, most of it describing an older version of the product, with three different refund policies live at the same time.

Volume without maintenance is worse than no documentation, because the agent has to pick between plausible answers instead of escalating cleanly.

Specificity beats coverage, which surprised us

We expected intent coverage to dominate, since it carries the most points. It didn't. Coverage alone correlated at r = 0.68. Answer specificity alone correlated at r = 0.74.

Having a document about a topic is worth less than having a document that resolves it. In the cohort, 62% of knowledge base articles described a policy without stating the rule inside it.

What the difference looks like in practice

A typical article reads: "Refunds are processed in line with our returns policy. Contact support if you have questions about your eligibility."

An agent can quote that. It cannot act on it, so every refund email escalates.

The version that works reads: "Refunds are approved automatically for orders under $150 returned within 30 days in original packaging. Orders over $150, or returned between 31 and 60 days, need supervisor approval. Nothing is refundable after 60 days."

Same policy. One of them is executable. Rewriting twenty articles this way moves the number more than writing two hundred new ones, and it's a fraction of the work.

Contradictions are the expensive failure

The median deployment had conflicting answers across documents on 9% of its top 50 intents. The worst had 23%.

Contradictions hurt twice. The obvious cost is escalation, when the agent detects the conflict and correctly refuses to guess. The worse cost is when it doesn't detect the conflict, picks the more recently updated document, and answers confidently with a policy the company abandoned eighteen months ago. That's how a well-behaved agent produces a wrong answer without any hallucination in the technical sense, which is a different problem from the one most teams prepare for when they think about preventing AI hallucinations in customer emails.

One deployment resolved contradictions across just its top ten intents and gained six points of autonomous resolution in three weeks. No model change, no new integrations. They deleted four documents and edited seven.

Freshness matters less than people assume

Freshness correlated at r = 0.52, the weakest of the four components. That runs against the standard advice to audit everything on a schedule.

The reason is that staleness only hurts where the underlying facts changed. A five-year-old article explaining how to reset a password is fine. A four-month-old article listing shipping cutoffs that moved last quarter is actively harmful. Blanket review cycles spend most of their effort on documents that were never going to drift.

Review by volatility, not by calendar.

Most of your knowledge isn't in your knowledge base

Scoring 42 knowledge bases turned up something we didn't set out to look for. In most support organisations, the help centre holds maybe half of what the team actually knows.

The rest sits in three places. Macros and canned responses hold the phrasing that has survived contact with real customers, often with the exact thresholds the help centre omits. Closed ticket threads hold the reasoning, including every exception a supervisor has approved. And a handful of senior agents hold the judgement calls that were never written anywhere, which is why the same three names get tagged in Slack all day.

Macros are the most undervalued of the three. A mature macro library is a knowledge base that has been continuously tested against real customers, and it usually scores higher on specificity than the official documentation does. If your help centre says refunds follow the policy and your top macro says "I've gone ahead and refunded your order since it's under $150 and within the return window," the macro is the document worth keeping.

Two practical consequences. First, don't start knowledge work by auditing the help centre. Start by exporting your top 30 macros and your last 500 resolved tickets for the intents you care about, because the answers are already written and just need consolidating. Second, when an intent has no document and no macro but does have a person everyone asks, that's your highest-priority gap. It's also the most fragile thing in your operation, since it walks out of the building whenever that person takes leave.

Teams that mine macros and ticket history before writing anything new typically lift their score faster than teams who start with a blank page, and the resulting documents need less correction because the phrasing has already been validated on real customers.

Where the curve flattens

The returns are steeply non-linear, which changes where you should spend effort.

• Below 55: each point of KB score is worth roughly 0.8 points of resolution rate. This is the highest-return work available to a support team.
• 55 to 80: about 0.5 points. Still worth doing, and still cheaper than most alternatives.
• Above 85: about 0.2 points. Going from 85 to 95 buys around two points of resolution, for considerably more than twice the work.

If you're at 45, knowledge work is the best investment on your roadmap. If you're at 88, stop. The constraint has moved somewhere else, usually to integrations or to policy.

What a better knowledge base can't fix

There's a ceiling, and knowing where it sits stops teams from grinding at documentation forever.

Between 18% and 22% of email volume in our cohort needed a system to be changed rather than a question to be answered. Cancelling a subscription, reissuing a shipment, updating a billing address. Perfect documentation gets an agent to "here's how cancellation works" and no further. Closing that gap takes write access, which is why integrations that let agents take action tend to be the binding constraint for teams who have already done the knowledge work.

Another 8% to 12% is genuinely not automatable at any knowledge score. Novel complaints, legal threats, requests that need commercial judgement. Those should escalate, and an agent that resolves them is a liability rather than an achievement.

Add it up and the practical ceiling for most inboxes sits between 78% and 84%. A knowledge base scoring 85 with full write access will get you there. Nothing gets you past it.

Moving 20 points in three weeks

The median deployment that lifted its score by 20 points did it with about six hours a week from one support lead, over three weeks. Not a documentation project. A focused pass.

1. Pull your top 20 intents by volume. Not by ticket category, by what customers actually write about. Most teams find their taxonomy and their reality have drifted apart.
2. Find the intents with no authoritative document. These are usually the ones where the answer lives with one experienced agent who everybody messages on Slack. Write those first.
3. Rewrite descriptions into rules. For every document covering a top-20 intent, check whether a new hire could execute it without asking a question. If not, add the thresholds and the conditions.
4. Delete rather than reconcile. When two documents conflict, resist the urge to merge them into a longer one. Pick the correct one, delete the other, and note the decision.
5. Fix your five most volatile articles. Pricing, shipping, eligibility, anything seasonal. Then put those five on a real review cycle and leave the rest alone.

Steps three and four account for most of the gain. They're also the two that get skipped, because writing new articles feels more productive than editing old ones.

Why this shows up in every deployment conversation

When two customers get different results from the same AI email agent, this is nearly always the reason. It's why we score the knowledge base during onboarding and validate the projected resolution rate against historical tickets before anyone signs, rather than quoting a single number to everybody.

Robylon lands in the 60 to 80% band across deployments, and where you land inside it is mostly decided before go-live by the two variables in this article: what your documents actually say, and what your agent is allowed to do. The week-by-week ramp to 70% resolution covers the timing side of that, and if you're starting the knowledge work now, our guide to building a knowledge base optimised for AI resolution goes deeper on structure and formatting.

The team at 48% didn't need a better model. They needed to spend eighteen hours editing documents, which is roughly what they'd been spending every week answering the same refund question by hand.

Ready to find out what your knowledge base is actually worth? Robylon AI resolves 60-80% of customer emails autonomously with agents that take action across Zendesk, Shopify, Stripe, HubSpot, and 60+ other integrations. Start free at robylon.ai