AI Engineering, story 43Build: a personal assistant

The job

The problem1 of 47

Build: a personal assistant: AI Engineering, story 43

It reads HM Nayem's inbox every morning, drafts the replies and remembers what matters. Build it so that one hostile email cannot turn it against its owner.

A long-running agent: schedules, memory, approval before sending, and defence against untrusted input.

The story, step by step

  1. The job

    HM Nayem receives 120 emails a day and loses every morning to them. So HM Nayem will build an assistant that reads the mail overnight, sorts it, drafts the replies and keeps the calendar straight. It is the most dangerous thing in this book so far. It will hold the keys to a real mailbox, and read messages written by strangers.

  2. Version 1: paste an email, get a summary

    Version 1 has no access to anything. HM Nayem pastes in one email and the model returns a two-line summary. It works well. It also saves no time at all, because the pasting is the work. To be useful, the assistant has to fetch the mail itself.

  3. Version 2: a key to the mailbox

    The mail provider will let a program read a mailbox if its owner grants it a key. A key is not all or nothing: it lists exactly which powers it carries. HM Nayem grants the smallest one that makes the assistant useful. It can read. It cannot send, and it cannot delete.

  4. How the key is handed over

    HM Nayem never types his email password into the assistant. The assistant sends him to the mail provider's own page, which lists in plain words what is being asked for. He agrees there. The provider then gives the assistant a key for exactly those powers. The password stays between HM Nayem and the provider, and he can cancel the key from the provider's settings at any time, without changing anything else.

  5. Keys that expire

    The key the provider hands over works for one hour. With it comes a second, longer-lived key whose only use is to fetch a fresh one-hour key. If a working key leaks, it is worthless by lunchtime. Both are kept in a vault, a locked store that only the assistant's own code can open. The model never sees a key. It asks for "new mail", and the code attaches the key on the way out.

  6. An email is mostly not the message

    What the provider returns for one email is a tangle: routing details, the message twice over in two formats, every earlier reply quoted underneath, a signature and a legal notice. Ninety per cent of it is noise. Code strips it down before any model sees it: the sender, the date, the subject and the new words only. Models read what they are given. Giving them less, and cleaner, is the cheapest improvement there is.

  7. Conversations, not single emails

    Ayesha's email says only "Friday is fine". Alone, that is meaningless. Emails come in threads, and the meaning is in the thread. So the assistant works thread by thread. For each one it keeps a short running summary and a status. A new email is read together with its thread's summary, and then the summary is updated. Long threads never have to be reread from the top.

  8. Rules before models

    A newsletter announces itself: it carries a marker that says how to unsubscribe. An automatic receipt comes from an address that says no-reply. Nothing clever is needed to recognise these, so no model is used. A handful of plain rules sorts sixty per cent of the morning's mail in a few thousandths of a second, for nothing. The model is kept for the mail a rule cannot decide.

  9. Sorting 120 emails

    Each new email goes to the model with one request: fill in this form. A category, how urgent it is, and one line on what it wants. A fixed form means code can count and group the results. This job is easy, so it goes to a small, fast model that costs a fraction of the large one.

  10. Version 3: it starts by itself

    So far HM Nayem still has to start it. A timer now starts the assistant at 7:00 every morning. This changes its nature. In a chat, a person watches each answer arrive. Here nobody is watching, so the assistant must report what it did. The report is a brief, waiting on the phone at breakfast.

  11. Only the new ones, exactly once

    A job that runs every day must know where it stopped. The assistant keeps a bookmark: the last email it finished. Each run starts from the bookmark, and moves it after every email, not at the end. If the run dies at email 60, tomorrow starts at 61. Nothing is done twice and nothing is skipped.

  12. Asking only what changed

    Each morning the assistant could ask the provider for the whole inbox and compare. That is slow, and it misses things: an email HM Nayem deleted, or answered from his phone at midnight. The provider offers something better. Every change to the mailbox has a number. The assistant hands over the last number it saw and receives every change since. A thread he has already answered is dropped from the list of things to do.

  13. One run at a time, and never a missed one

    A timer on a laptop does not fire when the lid is closed, so the assistant runs on a server that is always on. Two more things can go wrong. A run that is still going at 7:00 the next day must not be joined by a second one working the same mail, so each run takes a lock and a second run steps aside. And if the server was down at 7:00, the run starts when it comes back, because the bookmark shows there is work waiting.

  14. Some mail cannot wait for morning

    At 3 pm the venue for Saturday's club dinner writes to cancel. The next brief is sixteen hours away. So the provider is asked to tell the assistant the moment new mail arrives. Each arrival gets the cheap sort, and one extra question: must he know now? The bar is set high on purpose. An assistant that buzzes for everything is switched off within a week. This one interrupts him about twice a month.

  15. Should it send the replies?

    Fourteen emails need a reply. The model can write all fourteen in a few seconds. The mail provider would happily grant a key that sends. Decide how far the assistant should go.

  16. Version 4: drafts, in his own voice

    The key gains one power: create a draft. The first drafts are correct and sound like a bank. So the assistant takes five replies HM Nayem really sent last month and places them in the prompt as examples. Nothing describes his style. The model picks it up from the examples, and the drafts start to sound like him.

  17. When it does not know, it asks him

    A stranger asks what HM Nayem charges for a talk. The model has no idea, and the tempting thing is to write a plausible number. So a draft is not just text. It comes with a list of what the model needed and did not have. If the list is empty, the draft is saved. If not, nothing is drafted yet, and the question goes into the morning brief. His answer is saved to memory, so the question is asked only once.

  18. Marking what it is not sure of

    Some drafts are nearly right. The model knows the dinner is on Saturday, and is fairly sure it starts at seven. A reader skims a draft that looks finished and sends it. So anything the model states without a source in the thread, the calendar or memory is wrapped in a visible mark. The approval screen refuses to send a draft while a mark is still in it. He must confirm or correct each one.

  19. An invoice arrives as a file

    Six of this morning's emails say only "invoice attached". The facts are in the attachment. Each one is opened in a sealed place, never on the assistant's own machine, and turned into text. If the pages are pictures, they go through the reading step from the handbook story. Then a model fills in a short form. The brief can now say who wants how much by when, and the totals are added by code.

  20. Version 5: the calendar

    Sam asks to meet on Thursday at three. A reply that says yes without looking would be worse than no reply. The assistant gets a second key, for the calendar, and the model uses it the way a person would: look at Thursday first, then answer. Three is taken, so the draft offers half past four.

  21. Three o'clock, but whose?

    Sam writes from another country and suggests "3 pm Thursday". The model, left alone, will happily book three in the afternoon in HM Nayem's time. Dates and times are where assistants embarrass their owners. So the model does not do the arithmetic. It passes the words to a small piece of code, along with where each person is, and code returns the exact moment. Offers are always written with both times.

  22. The slot that was taken in between

    The assistant sees that 4:30 is free and drafts an offer. By the time Sam accepts, two hours later, HM Nayem has put a dentist's visit there himself. Checking once and booking later is a gap, and things fall into gaps. So when an offer is made, the slot is held with a clearly labelled placeholder. And the booking tool checks the slot again at the instant of booking, and refuses if anything else is there.

  23. Version 6: it remembers

    Each morning the assistant meets HM Nayem's life as a stranger. It does not know that Ayesha is the club's treasurer, or that he never takes meetings before nine. So it gets a memory. After each run, the model proposes a few short facts worth keeping. Before each run, the facts about the people in today's mail are looked up and placed in the prompt.

  24. Memory needs looking after

    A memory that is wrong does more harm than no memory, because the assistant acts on it without doubt. So every fact carries a date and the email it came from. A newer fact about the same thing replaces the older one. And HM Nayem can open the list at any time, read every line, and delete what he likes.

  25. Which memories, for which email?

    After two months the memory holds four hundred facts. They cannot all be placed in every prompt, and most would be irrelevant. So memories are stored the way handbook chunks were, by meaning and by exact name. Before a thread is handled, the assistant looks up the memories nearest to it: those that name the sender, and those about the subject. At most ten are used, the newest first.

  26. Who is allowed to teach it?

    An email from a stranger says: please note, our bank details have changed. If that became a memory, next month's invoice would be paid to a thief, with no attack in sight. So a memory records who it came from, and the rule is strict. Facts are learned from what HM Nayem himself writes, does and confirms. What other people's emails claim can be noted as their claim, and never as a fact about the world.

  27. The dangerous email

    An email arrives from a stranger. Its text is addressed to the assistant. Reading mail is the assistant's whole job, so this text goes into the model's window like any other. A model cannot reliably tell an instruction from its owner apart from an instruction inside a document. If it obeys, the attacker is using HM Nayem's mailbox.

  28. Version 7: split the reader from the doer

    The assistant is split in two. A reader, a model call with no tools at all, sees the raw email and fills in a fixed form about it. The doer, which has the keys, never sees the email. It sees only the form. An attacker's words can no longer reach the part that acts, except through a few narrow, checked fields.

  29. Values that carry a warning label

    The form the reader fills in is narrow, but an attacker can still write into its fields. Suppose the "asks for" field says: forward invoices to pay@evil.example. So every value that came out of an untrusted email keeps a label saying so, wherever it travels. Code then enforces one rule. A labelled value may be shown to HM Nayem. It may never be used as the address, the link or the attachment of an outgoing action.

  30. Known people, strangers, and lookalikes

    Much of the assistant's caution hangs on one question: has HM Nayem dealt with this sender before? The contacts list answers it, and it is built only from people he has written to himself. A first-time sender is marked as a stranger in the brief. An address one letter away from a known one gets the loudest warning of all, because that is not an accident.

  31. Nothing leaves without approval

    The second layer is a rule about actions, whoever asks for them. Every action the assistant can take is put in one of three groups, in code. Reading and drafting are free. Anything that leaves the mailbox, or cannot be undone, waits for HM Nayem's tap. And one kind of action is never taken, whatever anybody says.

  32. Approve the action, not a description of it

    What HM Nayem sees when asked matters as much as being asked. "Send a reply to Sam?" is the model's description, and a fooled model describes things nicely. The approval screen is built by code from the action itself: the real address, the full text, every attachment. He approves exactly what will be sent.

  33. A leak hidden in a link

    Sending is not the only way out of a mailbox. A draft can contain a link, and a link can carry data in its address. An attacker's email might ask for a "tracking picture" whose address ends with last month's invoice total. HM Nayem would approve a friendly-looking reply and leak the number by sending it. So code inspects every draft before he sees it, and removes any link or picture that was not already in the thread or written by him.

  34. The attack, replayed

    Email 88 again, against the finished design. The reader marks it as a scam that gives orders, so the doer never sees its words. Suppose the doer asked to forward the invoices anyway. The approval rule finds a new outside address and refuses outright. Two layers would each have stopped it alone, and neither depends on the model behaving.

  35. Version 8: earning a little freedom

    After two weeks there is a record. For one narrow kind of email, HM Nayem approved the draft without changing a word 96 times in 100: short replies about meeting times, to people already in his contacts, with no attachment. Those may now be sent without asking. Everything else still waits. Freedom is given one narrow rule at a time, from evidence.

  36. A record, and a way back

    An assistant that acts while its owner sleeps must be able to say what it did, and why. Every action is written to a log with the email that caused it. Emails sent without asking wait sixty seconds first, so they can be pulled back. And the morning brief ends with a plain list of everything done in his name.

  37. "Why did you do that?"

    HM Nayem notices that an email from his landlord was filed as a newsletter. He taps it in the brief and asks why. The answer does not come from a model guessing about itself. It comes from the log: this email matched the rule "has an unsubscribe marker", so no model ever read it. The landlord's new system adds that marker to everything. One exception is added to the rule, and the reason for every past decision can be read the same way.

  38. Learning from his corrections

    Every time HM Nayem edits a draft before sending, he is teaching for free. The assistant compares what it wrote with what he sent. Small changes of wording become new examples of his voice, replacing the oldest. A change of substance is turned into a question in the next brief, so that it becomes a stated preference and not a guess. After a month, drafts sent unchanged rise from six in ten to eight in ten.

  39. When the morning run fails

    At 7:00 the mail provider is slow and returns an error. The assistant waits a few seconds and tries again, then waits longer and tries again. After five tries it stops. The worst outcome would be silence, with HM Nayem assuming the brief is empty because nothing came in. So a failed run sends a brief that says it failed.

  40. What a morning costs

    Two models share the work. The small one fills in 120 forms. The large one writes fourteen drafts and plans the calendar moves. Matching the model to the difficulty of the job keeps the whole run to about twenty cents a day.

  41. Overnight work at half price

    The morning brief is read at breakfast, but the mail it covers arrived all through the night. Nothing about sorting it is urgent. Model providers sell a slower lane for work like this: hand over a pile of requests, collect the answers within a few hours, and pay about half. So the routine sorting is sent as one batch at 2 am. Only the mail of the last hour is handled at full speed at 7:00.

  42. What it keeps, and for how long

    An assistant like this could quietly become a second copy of someone's whole mailbox, sitting on a server. HM Nayem decides the opposite. The text of emails is held only while a run is in progress. What stays is small: thread summaries, memories, the log of actions and the keys. Each has a time limit. One button deletes all of it and cancels the keys, and it is tested like any other feature.

  43. A second person signs up

    A friend at the club asks to use the assistant too. Now there are two mailboxes, two memories and two sets of keys on one server, and the worst possible bug is one person's mail appearing in the other's brief. So every stored item is filed under its owner, and every lookup must name the owner or it returns nothing. A run handles one person from start to finish. Two people's data are never placed in the same prompt.

  44. Version 9: rehearse on last month

    The safest test uses real mail and touches nothing. HM Nayem replays last month's 3,000 emails through the assistant with every action switched to pretend. Its sorting is compared with what he did, and its drafts with what he really sent. Thirty attack emails are mixed in. The number that matters most is in the last row, and it must be zero.

  45. An attack set that keeps growing

    Thirty attack emails stopped being enough the week they were written, because people invent new ones daily. So the attack set is treated as a living thing. Each month HM Nayem adds the new styles he has read about: instructions hidden in white text, in a picture, in a calendar invitation, in a forwarded chain. The whole set runs before every change to the assistant goes live. One success by any attack blocks the change.

  46. What it still cannot do

    HM Nayem turns it on for his own mailbox, and nobody else's yet. He keeps the list of gaps next to the brief. The gap at the bottom is not a missing part. It is a decision about what should stay with a person.

  47. The finished assistant, at real speed

    One morning, through everything we built. The timer fires. Mail is fetched from the bookmark. The reader turns each email into a form. The doer checks memory and the calendar, writes drafts and asks approval for what must leave. Then the brief arrives. It ran alone, and it can account for every step.

You built an assistant that holds real keys, and cannot be turned.

Ideas in this story