AI Engineering, story 18The anatomy of a harness

One sentence in, twelve emails out

1 of 16

The anatomy of a harness: AI Engineering, story 18

One sentence goes in and twelve emails go out. Slow the harness right down and name its ten parts, one at a time.

A harness is a loop, a prompt builder, tools, a dispatcher, permissions, a context manager, saved state, a record, limits and notes.

The story, step by step

  1. One sentence in, twelve emails out

    HM Nayem types one sentence, and a minute later twelve members have a polite reminder in their inbox. Between the sentence and the emails sits a program called a harness. Earlier stories showed what a harness does. This one takes it apart, to show what it is made of.

  2. The model is one call

    Start with what the harness is not. The model is a service that turns text into more text. It cannot open the members list, send an email or remember the last round. Every other thing in this story is ordinary code that HM Nayem could read line by line. There are ten parts.

  3. Part 1: the loop

    The centre of every harness is a loop of six lines. Call the model. If it asks for a tool, run the tool and add the result to the conversation, then call the model again. If it asks for nothing, stop. The loop is what turns a model that answers once into an agent that keeps working.

  4. Part 2: the prompt builder

    Before each call, something has to assemble the text the model will read. The prompt builder does it, in the same order every time: fixed instructions, the list of tools, any notes about this club, then the conversation so far. The last story was about this part alone.

  5. Part 3: the tool registry

    The harness keeps a list of every tool it offers. Each entry holds four things: a name, a description for the model to read, the actual function to run, and how dangerous the tool is. The first two go into the prompt. The last two never leave the harness.

  6. Part 4: the dispatcher

    The model replies with a request: find the members who have not paid for June. The dispatcher takes it from there. It checks that the tool exists and the inputs have the right shape, runs the function with a time limit, and wraps whatever comes back, a result or an error, as the next piece of the conversation.

  7. Part 5: the permission check

    Now the model asks to send an email. The registry marks that tool as one that acts, so the dispatcher does not run it. It hands the request to the permission check, which holds it and asks HM Nayem. He sees the real email and approves all twelve. Whether to ask is decided by a rule in code, never by the model.

  8. Part 6: the context manager

    Every round adds to the conversation, and the model's window has a limit. The context manager watches the size. It shortens long tool results before they are added, and when the conversation nears the limit it replaces the oldest rounds with a summary. Without this part, every long job would end in an error.

  9. Part 7: the saved state

    After the seventh email, the machine running the harness restarts. A harness that kept the conversation only in its own memory would begin again and email seven people twice. So after every round, the conversation and a note of what has been done are written to a store. On restart the harness reads them back and carries on from email eight.

  10. Part 8: the record

    Separately from the state, the harness writes a record of everything that happened: each model call, each tool run, each decision of the permission check, with times and costs. Nothing reads it while the job runs. It exists for the day HM Nayem asks why something happened.

  11. Part 9: the limits

    A loop needs a way to end badly as well as well. The limits are a few numbers set in code: the most rounds, the most money, the most time. A confused model will go round for ever if nothing stops it. When a limit is reached, the harness stops and reports exactly how far it got.

  12. Part 10: the notes

    The last part is the one that makes a harness feel as if it knows you. Before the first round, it loads a short file of standing notes: how this club writes, who the treasurer is, what was learned last time. The model starts every job as a stranger. The notes are how the harness introduces it.

  13. Whose job is it?

    With the ten parts named, it should be clear who does what. Try one.

  14. The same ten parts, everywhere

    A chat assistant, a coding agent and a voice assistant look nothing alike, and are built from the same ten parts. What differs is the tools in the registry, the rules in the permission check and the limits. Once you can see the parts, every AI product in this book becomes a variation on one design.

  15. A thin harness, or a thick one

    One choice runs through every part: how much does the harness decide, and how much is left to the model? A thick harness scripts the steps and asks the model only to fill gaps. A thin one supplies tools and rules and lets the model find the way. As models improve, harnesses get thinner, and the rules stay.

  16. All ten parts, at real speed

    The whole job once more. Notes in, prompt built, the model asks, the dispatcher runs, the permission check asks HM Nayem, the emails go, the state is saved and the record is written. The model was called three times. The harness did everything else.

A harness is ten small parts of ordinary code.

Ideas in this story