AI email agent
An agent that answers the venue inbox. It polls every ten minutes, classifies each message, and writes a reply into the Drafts folder for a human to read and send. It has never sent an email and cannot. Everything it isn't certain about goes to a person instead.
The guard layer
The model proposes a reply. Then ordinary code decides whether it is allowed to exist. A prompt is a request; code is a rule — and the model cannot talk its way past a rule.
- No bank details, ever. Every draft is scanned for sort-code, account-number and IBAN patterns. The highest-value rule in the system, because invoice-interception fraud targets hospitality inboxes specifically.
- Price whitelist. Every currency figure in a draft must already appear in the knowledge base. An invented price cannot leave the building.
- Force-escalate keywords read from the incoming email, not the reply — refunds, chargebacks, solicitors, environmental health, allergy, coeliac, anaphylaxis. These escalate regardless of what the model concluded.
- Link whitelist, competitor blocklist, a length ceiling, and a block on any draft that discloses being automated.
- A confidence floor: below 0.75 it escalates whatever it said.
What I built
- A six-stage pipeline where stages two and four contain no AI at all. A deterministic pre-filter drops booking platforms, suppliers, marketing and anything from a no-reply address before a single token is spent; the guard layer sits between the model and the mailbox.
- An 18-section knowledge base as one editable file per venue, read fresh on every run. Changing a price needs no redeploy, and if a fact isn't in the file the agent doesn't know it and escalates.
- Application-level mail permissions clamped to a single mailbox by an Exchange Application Access Policy, rather than granting tenant-wide access. Verified both ways: the intended mailbox granted, another explicitly denied.
- Every decision logged to SQLite with its token cost. Measured, not estimated: £0.019 for twenty messages, around £1 a month at real volume.
- Built multi-venue from the start — a second site needs a folder and a filled-in questionnaire, not a code change.
What went wrong
- The first version over-escalated badly. The classifier treated any mention of a room as an availability enquiry, so questions about hairdryers, breakfast and luggage went to a human. The distinction is whether they're asking if a room is free — everything else the knowledge base covers is just a question. Fixed by splitting the intent taxonomy with explicit examples.
- Documented rather than hidden: the schedule only runs while the machine is awake. Acceptable while a human sends every draft, and a blocker that must be cleared before auto-send is ever switched on.
What the analysis found
Exporting twelve months of mail was meant to size the problem. It answered a different question: the inbox gets around 195 genuine guest threads a year, roughly four a week, already answered 80% of the time. The gain was never hours saved — it was replies going out in minutes instead of days, including at 11pm on a Sunday, and being right every time.