Guides, comparisons and field notes on AI agents that resolve support conversations end to end, across email, chat, voice and WhatsApp.
CSAT and first response time were designed for human-only support. They quietly break once AI starts resolving most of your email volume. This deep dive covers the eight metrics that actually tell you whether your AI email agent is working, and how to wire them into a measurement stack you can trust.
When a customer asks why your AI sent them the wrong answer, most support teams cannot say. This deep dive covers the trace model for email agents: what a span is, which attributes to record, why reason codes matter more than confidence scores, and how to reconstruct any ticket end to end.
LLM-as-judge is the only practical way to score support reply quality at volume, and it fails in patterned ways. Covers verbosity and position bias, calibration against human labels with Cohen's kappa, judge drift, and where humans stay in the loop.
A golden set is the fixed benchmark every change to an AI email agent should clear before it reaches a customer. Covers stratified sampling, labelling outcomes rather than reply text, what to actually measure, and how golden sets rot.
Prompt changes fix one thing and quietly break three others. This is the engineering playbook for regression testing an AI email agent: turning production incidents into permanent test cases, writing assertions that survive paraphrase, and wiring the suite into CI so a deploy can actually fail.
A tribunal rejected the argument that a chatbot is a separate legal entity, and the reasoning has held ever since. This is a practical read on where liability sits for AI customer service in 2026, what the EU and US rules changed this year, and the controls that make up a defensible record.
Most AI governance committees die of vagueness within two quarters. This playbook covers the six seats that matter, the four approval gates worth owning, a monthly agenda that takes 45 minutes, and the decisions the committee should deliberately leave alone.
Most human-in-the-loop programmes in customer service are approval queues where reviewers approve almost everything. This is a practical argument for designing oversight around risk-weighted sampling and real reviewer authority, plus the metrics that show whether it is working.
Procurement questionnaires have started asking about ISO 42001, and the instinct is to forward it to whoever owns SOC 2. That is mostly wrong. Here is what the standard certifies, which controls land inside a support inbox, and what a vendor certificate does and does not cover.
Security questionnaires increasingly ask AI support vendors for model documentation, and most responses are marketing PDFs. This is a working template for a support AI system card, covering what the system is, what it's allowed to do, how well it works, and how it fails.
Security questionnaires ask about NIST AI RMF alignment and get a paragraph about commitment to safety in return. Nobody wanted a paragraph. Here is what Govern, Map, Measure and Manage look like inside an email queue, and the six artefacts that answer the question properly.
Saying you disclose is easy. Proving that one customer, on one date, was told an AI was answering is a different problem, and most support stacks cannot do it. Here is the evidentiary chain a regulator will actually ask for.
Article 50 makes you disclose that an AI is answering. It does not tell you how to word it. Here are 15 disclosure lines that work in a support inbox, the phrasings that reliably backfire, and how to test the change without guessing.
Chat you can take offline. Email is already in the inbox. This is the hour-by-hour runbook for the first 24 hours after an AI email agent sends something it should not have — kill switch, blast radius, correction, notification, and root cause.
Article 50 of the EU AI Act became enforceable on 2 August 2026 and covers every customer-facing AI agent, not just high-risk ones. Here's what disclosure actually requires in a support inbox, who carries the duty, and where the rules are still genuinely unclear.
The pretext used to be aimed at a tired rep on a night shift. Now it is aimed at a model with a very different set of weaknesses. What attackers try, why knowledge-based verification stopped working, and how to design checks an agent cannot be argued out of.
An AI email agent reads untrusted content, holds private data, and can send externally. That combination is an exfiltration path by design. Here is what the attack patterns look like in a real inbox, and which controls actually reduce the blast radius.
Most email agents are one box with one credential, so a single malicious message reaches everything the agent can reach. The stronger defence is architectural: the component that reads untrusted email should not be the component that can change anything.
A support agent reads mail from strangers, holds access to customer data, and can send email externally. That combination is the whole security problem. This is the architecture that contains it, layer by layer.
Support teams harden the email body against injection and leave the attachment untouched. It is the larger surface. This is where instructions hide in PDFs, screenshots, and spreadsheets, and what to do about it.
Most teams test their AI email agent for accuracy and never test it for adversity. This is the suite to run before launch: seven attack categories, how to write and score each case, and where to fix what you find.
Security teams already have a framework for why AI agents get compromised: the lethal trifecta of untrusted content, private data access, and external communication. An email support agent has all three by definition. This explains what that means operationally and which leg to constrain on each workflow.
Your support inbox is an open port that any stranger can write into, and an AI agent reads every word of it as potential instruction. This is what prompt injection looks like in a support queue, why filters don't close it, and the architectural controls that actually bound the damage.
The support contact form has survived two decades without changing shape, and it is about to stop making sense. An argument for agent-first entry points: what replaces the form, what breaks when it goes, and the five things worth doing in the next year.
The Agent Payments Protocol shipped as a formal A2A extension in April 2026, and refund email is where it lands first. This explains the three mandates in support terms, what changes about proving authorization, and the new dispute class nobody has a playbook for yet.
