NIST AI RMF Applied to Email Support Automation

“Describe your alignment with the NIST AI Risk Management Framework.”

It's one line on a security questionnaire, and the answer that comes back is almost always a paragraph about how seriously the company takes AI safety, followed by a mention of human oversight. Nobody who wrote the question wanted a paragraph. They wanted to know whether you can name the artefacts.

Because the framework isn't a philosophy. It's four functions broken into categories and subcategories, and each subcategory implies a document, a metric, or a process that either exists in your support org or doesn't.

What the RMF is, and what it stubbornly isn't

NIST published AI RMF 1.0 as NIST AI 100-1 on 26 January 2023, under a mandate from the National AI Initiative Act of 2020. As of August 2026 it is still the current version. NIST has said the document is being revised; no finalised 2.0 exists.

That confuses people searching for it, because the framework does get extended — through Profiles, not version numbers. The Generative AI Profile, NIST AI 600-1, was finalised on 26 July 2024 and applies the same four functions to generative and large language model risk. It's the document most people mean when they ask about a 2024 version, and for anyone running an LLM in a support inbox it's the more directly useful of the two.

Three properties worth being blunt about. It's voluntary. It's non-certifiable — nobody can audit you against it and you self-claim alignment. And the framework itself runs about 40 pages of principles, while the companion Playbook runs well over 140 pages of suggested actions and transparency questions. If you only read the framework you get abstractions. The operational content is in the Playbook.

Why a voluntary framework keeps showing up in your contracts

Here's the bit that surprises support leaders. The RMF has no legal force, and it's turning up in more procurement language every quarter.

The mechanism is federal, and it survived a change of administration. Executive Order 14110 was rescinded in January 2025 and replaced by EO 14179. That rescission didn't unwind the downstream layer. OMB replaced its earlier memos in April 2025 with M-25-21, covering agency use of AI, and M-25-22, covering acquisition — both of which retain the risk-management posture and the Chief AI Officer structure, and both of which reference NIST guidance for the relevant impact level. NIST publications are NIST products; they persist regardless of executive-order churn.

M-25-21 defines a category called High-Impact AI, meaning systems whose outputs serve as the principal basis for decisions with legal, material or significant effects. Most customer support automation sits below that line. But the definitions leak outward, and the acquisition language leaks faster.

A proposed GSAR rule published in June 2026 goes as far as defining an “LLM System Integrator” by reference to the AI Design and AI Deployment tasks in AI RMF 1.0, Appendix A — selecting models, setting system prompts, choosing retrieval sources, setting human-review thresholds. That is an exact description of what a support team does when it configures an email agent, written into procurement regulation.

Enterprise buyers borrow federal language. If you sell to anyone who sells to the government, the question reaches you eventually.

GOVERN: the function support teams skip

GOVERN is the one that cuts across the other three, and it's the least technical. It's also where most support orgs have the largest gap, because the answers exist in people's heads and nowhere else.

Four questions. If you can't answer them from a document, that's your finding.

• Who owns the agent? Not who manages the vendor relationship. Who is accountable for its behaviour toward customers, and does that person have the authority to switch it off without asking a VP at 3 a.m.?
• What is it categorically not allowed to do? Written down, approved, and versioned. Our guidance on when to resolve versus route to a human works through where those lines usually belong.
• How does a concern get raised? An agent who notices the AI mishandling refund threads needs a route that isn't a Slack message that scrolls away.
• What's the review cadence? Quarterly is fine. Never is a finding. “When something goes wrong” is also never, dressed up.

None of this requires tooling. It requires somebody to make decisions and write them down, which is harder than it sounds when the agent is already running and working.

MAP: your intent taxonomy is already half the artefact

MAP establishes context for a specific system — what it's for, where it operates, who it affects, what could go wrong. Support teams tend to think they have nothing for this. They usually have most of it under a different name.

Your intent taxonomy is the MAP artefact. The list of ticket types the agent handles, what each one does, which systems it touches, and what happens if it gets one wrong. What's typically missing is the last column.

Take a WISMO query and a subscription cancellation. Same agent, same inbox, wildly different failure profiles. A wrong shipping estimate produces an annoyed customer and a reopened ticket. A wrongly executed cancellation produces a churn event, a refund dispute and possibly a chargeback. The RMF wants you to have noticed that difference before deployment rather than after.

The exercise that produces the most value per hour: sit with your top 20 intents by volume, and for each one write the worst realistic outcome of the agent handling it incorrectly. Not the worst conceivable outcome — the worst one that could plausibly happen on a Tuesday. Then check whether your autonomy settings match the answers.

They often don't, and the mismatch is rarely in the direction you'd expect. Teams over-restrict low-stakes intents because those are the ones customers complain about loudly, and under-restrict the quiet high-stakes ones because nobody has complained yet.

MEASURE: where the whole thing usually falls apart

This is the weakest function in most support deployments, and it's weak for a specific reason: the metrics teams already have are business metrics, and MEASURE is asking about trustworthiness.

Resolution rate tells you how much volume the agent absorbed. It tells you nothing about whether the absorbed volume was handled correctly. A 78% resolution rate with a 12% reopen rate is not a good number wearing a bad hat; it's a worse number that hasn't been calculated yet.

What MEASURE actually wants, translated into inbox terms:

• A golden set. A few hundred real tickets with human-verified correct outcomes, held out and rerun on every material change. Without one, you have no way to distinguish a model improvement from a lucky week.
• Correctness segmented by intent. Aggregate accuracy hides everything worth knowing. The agent that's 96% right on order status and 61% right on billing disputes reports as 89% and looks fine.
• Reopen rate as a first-class metric, not a footnote under CSAT. It's the closest thing support has to a false-positive rate.
• Escalation quality, in both directions. Cases escalated that shouldn't have been, and cases resolved that should have escalated. The second number is the one nobody measures and the only one that carries real risk.
• Calibration. When the system reports high confidence, is it actually right more often? A confident wrong answer is worse than an uncertain one, and a model whose confidence doesn't track correctness makes every downstream threshold meaningless.

One honest warning on automated evaluation. LLM-as-judge scoring is the practical way to run correctness checks at volume, and it drifts. Judges are lenient toward outputs that read fluently, which is precisely the failure mode you're hunting — a well-written wrong answer. Calibrate the judge against human labels on a sample, periodically, and treat an uncalibrated judge as an unmeasured system rather than a measured one.

Most of the surrounding measurement discipline is covered in more depth in our piece on the email support metrics that matter, and the failure modes worth watching for in preventing hallucinations in customer email.

MANAGE: allocation, response, and the part everyone forgets

MANAGE is about acting on what the first three functions found — prioritising risks, treating them, responding to incidents, and eventually retiring the system.

The response half maps onto ordinary operational work. Escalation paths, kill switch, customer notification, root cause. If you've built an incident response runbook for the email agent, you have already satisfied most of it, and the evidence a reviewer wants is the runbook plus proof it's been tested.

The forgotten half is change management and decommissioning. MANAGE 4.1 asks for post-deployment monitoring, appeal and override mechanisms, decommissioning procedures and change management. Support teams handle the first two by instinct and the last two almost never.

Ask yourself what happens when your vendor deprecates the model version your agent runs on. Does anything rerun? Does anyone check the golden set before the new version handles live tickets, or does it just quietly start answering differently on a Thursday? That silent-substitution scenario is the most common real-world manifestation of a MANAGE gap, and it costs nothing to fix and almost nothing to notice you haven't.

Do them in order, and the order isn't obvious

Engineering-led teams start with MEASURE, because measurement is the fun part and there's tooling for it. That's backwards, and it produces a lot of dashboards nobody acts on.

Start with GOVERN and MAP. Until somebody owns the agent and the intent-level risk map exists, you don't know which numbers matter, so you end up measuring what's easy to instrument rather than what carries risk. We've watched teams build a careful evaluation harness for response tone while the refund workflow ran unbounded, and the harness was genuinely good work aimed at the wrong thing.

The sequence that works: name the owner, write the boundaries, map the intents, then decide what to measure, then build the runbook for when the measurement goes red. Roughly a week for the first two if the right people are in the room, and the first two are the ones that change behaviour.

Where the framework is genuinely unhelpful

Three limits, stated plainly, because uncritical framework advocacy is its own kind of risk.

It tells you to measure and never tells you what good looks like. There are no thresholds anywhere in it. That's deliberate — context varies enormously — but it means a team can be fully aligned with MEASURE while running an agent that's 70% accurate on billing, because they measured it, documented it, and accepted it. Alignment is not quality.

It was written for AI systems broadly, and support email has specifics the framework doesn't reach. Inbound email is untrusted content by construction, which makes prompt injection a first-class threat that the RMF handles only at the level of “secure and resilient.”

And self-attestation is worth exactly what it costs. Since nobody audits you, an RMF alignment claim carries no external assurance at all. That's the gap ISO 42001 fills as the certifiable counterpart, and increasingly the reason enterprise questionnaires ask about both.

The artefact set that answers the question

Circling back to the questionnaire. A good answer isn't prose about your commitment to safety. It's a short list of things you can attach:

1. An AI system inventory covering every model touching customer conversations.
2. An intent-level risk map with the worst realistic failure per intent.
3. A written autonomy policy naming what never auto-sends, with an owner and a version.
4. An evaluation report: golden set size, correctness by intent, reopen rate, escalation quality, last run date.
5. An incident runbook, plus the date it was last exercised rather than last edited.
6. A change management note covering model version updates and what reruns when one lands.

Six documents. A support ops lead who already knows the answers can assemble them in a fortnight. The reason most teams don't is that nothing forces the issue until a deal is three weeks from close and the questionnaire is blocking it.

How this maps to Robylon

Robylon's email agent is built so that several of these artefacts are configuration rather than archaeology. Autonomous-action boundaries and per-workflow permissions are set explicitly during deployment, which means the autonomy policy and the running system are the same object. Escalation decisions and reasons are recorded as structured events, so escalation quality is a query rather than a research project. Every action taken across 60+ write-access integrations carries the retrieved context that produced it, and human review is logged distinctly from ordinary ticket handling.

Resolution runs at 60–80% autonomously, and during onboarding that number is validated against your historical tickets rather than asserted — which is, in RMF terms, a MEASURE activity that happens before deployment instead of six months after it.

The framework's real contribution isn't the four functions. It's that it gives a support team permission to ask the awkward question in a meeting: what happens when this is wrong, and who finds out?

Ready to run an email agent you can evidence as well as evaluate? Robylon AI resolves 60–80% of customer emails autonomously with AI agents that take action across Zendesk, Freshdesk, Shopify, Stripe and 60+ other integrations. Start free at robylon.ai