Build: a coding agent: AI Engineering, story 41
Start with a model that can only talk about code. Add one tool at a time until it can read, edit, run and check its own work.
The agent loop, built from nothing: tools, a sandbox, permissions, a trimmed context and helpers.
The story, step by step
The job
A coding agent is a program you give a task in plain words. It reads the code, changes it, runs it and checks its own work, round after round, until the task is done. HM Nayem will build a small one from nothing. It will take about 300 lines, and the model is not one of them.
Version 1: a model that can only talk
Version 1 sends the task to a model and prints the reply. The reply names a file, a line and a fix, with total confidence. There is no such file. The model has never seen this project, so it described the kind of code that usually has this kind of bug.
The loop, before any tools
Here is the idea that turns a chat into an agent, in six lines. Call the model. If its reply asks for a tool, run the tool, add the result to the conversation and call the model again. If the reply asks for nothing, the job is done. Everything else in this story is a tool to offer, or a rule around this loop.
Version 2: eyes
The first two tools let the agent look. One lists the files in a folder, the other returns the contents of one file. Each is a small function plus a description the model reads. The model starts where a new developer would: it asks what is in the project.
What a tool is, exactly
It is worth seeing the real shapes once, because every agent in the world uses them. A tool is defined by three things: a name, a description, and the list of inputs it takes. The model's request names the tool, gives the inputs and carries a ticket number. The harness's answer carries the same ticket number, so that the model can match each result to its request.
The first message nobody typed
Before the task, the model is sent a long message that HM Nayem never types: the system prompt. The harness builds it fresh at the start of every session from several sources. Some of it is fixed, such as how the agent should behave. Some is gathered on the spot, such as today's date, the folder it is in and the state of the project. A model told where it is makes far fewer wrong guesses.
Reading, one round at a time
Now it asks to read checkout.ts. Notice what the harness sends each round: the whole conversation again, with the newest result at the bottom. The model remembers nothing between calls. This growing list is the agent's entire memory of the job, and it will become a problem later.
Several tools in one round
The model wants to read three files. Asking for them one at a time would take three rounds, each resending the whole conversation. So a single reply may hold several requests, and the harness runs them side by side. There is one rule. Tools that only look may run together. Tools that change something run one at a time, in order, because two edits to one file at the same moment would collide.
Search beats reading
Reading file after file would take hundreds of rounds in a project this size. So the agent gets a third tool: search for an exact word across every file. This is how engineers find code too. One search for "coupon" returns six lines in three files, and the bug is in plain sight.
It can see the bug
The model has found the fault: the coupon's value is taken off twice. It has tools to list, read and search. It has no tool that changes anything. Decide what it will do next.
Version 3: hands
The edit tool does not let the model rewrite a file. It takes three things: a file, a piece of text that is in it now, and the text to put in its place. The harness makes the swap only if the old text appears exactly once. Small swaps are cheap to produce, easy for a person to review, and impossible to apply to the wrong place.
An edit that fails, usefully
The model tries the same swap in giftcard.ts, from memory, and gets the spacing wrong. The old text is not found, so the harness changes nothing and sends back an error. An error is just another result. The model reads it, reads the file again, and sends an edit that matches. A good harness makes failure loud and harmless.
Version 4: running things
The agent has changed two files and has no idea whether it worked. It needs to run the project's tests, which means running commands, and a command can do anything at all. So commands never run on HM Nayem's laptop. They run in a sandbox: a sealed copy of the project, with no internet and nothing else in reach.
What the sandbox is made of
"Sandbox" is one word for three separate walls, each put up by the operating system. The first wall is around files: the agent may write inside the project folder and nowhere else. The second is around the network. The third limits how much memory, processor and time it may use. A program inside cannot take these walls down, because it never had the right to.
The network is a short list
A sandbox with no network at all cannot install the packages a project needs. A sandbox with an open network can send the project's code anywhere. So all traffic from the sandbox passes through a gatekeeper outside it that holds a list of allowed addresses. The package registry is on the list. Nothing else is. A command that tries any other address fails, and the attempt is logged.
How a command is really run
The tool for running commands is the most powerful one, so here is exactly what it does. The harness starts a shell, the same program a developer types into, inside the sandbox. It hands over the command as text. Then it collects three things: what the command printed, what it printed as errors, and a number called the exit code, which is zero when the command succeeded. All three go back to the model as the result.
A shell that remembers
If every command started in a brand-new shell, the agent could change into a folder and find itself back at the start on the next command. So the harness keeps one shell open for the whole session. The folder it is in, and the settings it has made, carry over from one command to the next. The harness also tells the model which folder it is in after every command, because models lose track.
Commands that never finish
Some commands are not meant to end. Starting the website's server runs until someone stops it. A command that asks "are you sure? y/n" waits for a keyboard that is not there. Either would freeze the agent for ever. So every command has a time limit. Long-running ones are started in the background and given a number, and the model can ask later what that job has printed, or stop it. Commands are always run with the flags that answer questions in advance.
It finds out it was wrong
The tests run: 47 pass and one fails. The failure is a test that expected the old, wrong gift card total, written by someone who copied the bug. The model reads the output, corrects the test, and runs everything again. Forty-eight pass. An agent that can run its work can discover its own mistakes. That is the whole difference from autocomplete.
A result that is too big
Not every command is so brief. Building the site prints nine thousand lines. If the harness added all of that to the conversation, one result would use most of the model's window. So every result is cut to a limit: the start, the end, and a note saying how much was left out and how to ask for more.
Version 5: asking before acting
The sandbox is a copy. Changes to the real project, and commands with effects outside it, are a different matter. Every action now passes a permission check that sorts it into one of three kinds. For the middle kind, the harness stops and asks HM Nayem, who can say yes once, yes always, or no.
Rules are patterns, written down
The three kinds of action are not decided case by case. They are written as patterns in a settings file, and each pattern names a tool and what it is used on. Rules come from three places: the company, the project and HM Nayem. When rules disagree, the stricter one wins: a "never" beats an "ask", and an "ask" beats an "allow". Anything that matches no rule at all is treated as "ask".
Reading a command before running it
"Allow npm test" sounds safe. But a shell lets several commands be joined into one line, and a line that begins with npm test can end with anything. So the check does not compare the start of the text. It takes the line apart the way a shell would, at every join, and checks each piece against the rules by itself. One piece that is not allowed stops the whole line.
The file it must never read
One file in the project holds the shop's payment key. Reading looks harmless, but everything the model reads is sent to the model's provider and can turn up later in its output. So the check refuses a short list of paths outright. The model receives an error, like any other failed tool, and works around it.
An order hidden in a file
The agent opens a README that someone has tampered with. It contains a line addressed to AI agents, telling them to download and run a script. A model can be taken in by this, because to a model all text looks alike. The design assumes it will be. The sandbox has no internet, and the check would ask first anyway.
How much to ask: four modes
Asking before every edit is safe and slow. Asking about nothing is fast and frightening. So the harness has modes, and HM Nayem switches between them with one key. The mode changes how the "ask first" group is treated. It never changes the "never" group. The fullest mode is only offered inside the sandbox, where the walls do the protecting.
Hooks: rules the model cannot forget
The team wants every edited file tidied by the formatter. They could ask for it in the notes, and the model would remember most of the time. Most is not enough. A hook is a small program that the harness itself runs at a fixed moment: before a tool, after a tool, or when the agent says it is finished. The model does not choose to run it and cannot skip it. A hook can also say no.
Every edit can be undone
An agent will sometimes go confidently in the wrong direction for ten rounds. So just before each edit, the harness quietly saves a copy of the file as it was. HM Nayem can rewind the project to any earlier point in the session, and the conversation with it. On top of that, the agent works on its own branch of the code, never on the shared one, and saves its progress there at each milestone.
Version 6: the window fills up
A bigger task: move every price in the project from dollars to cents. After sixty rounds the conversation is nearly as large as the model's window, and most of it is old file contents nobody needs. The harness makes room in two ways. Old tool results are replaced by one-line stubs, and when that is not enough, the early rounds are replaced by a summary the model writes itself.
Why round sixty costs like round one
Each round resends the whole conversation, so the tokens sent grow with every round. That would make long jobs ruinous. But almost all of each request is identical to the one before: the same system prompt, the same tools, the same earlier rounds. The model's provider keeps that opening ready and charges about a tenth for reading it again. The harness's job is never to disturb the opening. Even changing the date in the system prompt would throw the saving away.
Plan first, touch nothing
For a big job, the worst mistakes are made in the first minute, by starting in the wrong place. So HM Nayem puts the agent in plan mode. It may read and search as much as it likes, and change nothing. It ends by writing a plan: what it found, which files it will change and in what order, and how it will know it worked. HM Nayem reads the plan, corrects one step, and only then lets it begin. A wrong plan costs a minute to fix. A wrong change costs an afternoon.
A plan that survives
A summary loses detail, and on a long job the agent can forget what is left to do. So it gets one more tool, which changes nothing in the project: a to-do list it writes and ticks off. The harness shows the list again in every round, outside the part that gets summarised. The agent's plan now outlives its own memory.
Version 7: notes for next time
Every session, the agent starts knowing nothing about this project, and relearns the same facts the slow way. So the project gets a notes file, written in plain words, which the harness places at the top of every conversation. HM Nayem writes most of it. The agent may suggest a line when it learns something the hard way.
Tools from somewhere else: MCP
The tasks the agent works on live in the team's issue tracker, a product made by another company. HM Nayem could write tools for it by hand. There is a better way. The tracker's makers publish a small server that speaks a standard language called MCP. The harness connects, asks "what tools do you have?", and receives names, descriptions and inputs in the very shape it already uses. They join the tool list. One connection, and the agent can read a bug report and mark it fixed.
A connected server is a stranger
Two cautions come with every connection. First, a server's tool descriptions and results are text written by someone else, and they go into the model's window. They are treated like a web page: useful, and not to be obeyed. Each server's tools get their own permission rules. Second, tools are not free. Forty tools from five servers cost twelve thousand tokens in every request. So the harness sends only the names at first, and fetches a tool's full description when the model asks for it.
Version 8: a helper with a clean window
Some side jobs need a lot of reading and give back very little, such as finding every place money is shown on screen. Doing it in the main conversation would fill the window with thirty files. Instead the agent starts a helper: a second loop with its own empty window and read-only tools. The helper reads everything and returns ten lines. Only those ten lines enter the main window.
Starting a helper, step by step
"Start a helper" is itself a tool the model can ask for, and what the harness does next is worth seeing in order. It opens a new, empty conversation. It gives that conversation its own system prompt, with a narrow role, and a smaller set of tools. The written brief becomes the first message. Then it runs the same loop as always, with its own limits. When the helper's loop ends, its last message returns to the main agent as an ordinary tool result.
Several helpers, and no collisions
Two parts of the big job do not touch the same files: the emails, and the reports. The agent starts two helpers that may edit, both at once. If they shared one folder, each would trip over the other's half-finished changes and run tests against a broken mix. So each helper gets its own complete copy of the project, on its own branch. When both have finished and their tests pass, the main agent brings the two branches together and runs everything once more.
A second agent reads the work
The agent that wrote a change is the worst judge of it. It has spent sixty rounds convincing itself. So before reporting, the main agent starts one more helper with an empty window and one job: read this change as a stranger would, and try to find what is wrong with it. It is given the change and the task, and none of the reasoning. It finds a refund path where prices are still in dollars.
Knowing when to stop
The loop ends by itself when the model asks for nothing more. That is the good ending. A harness also needs bad endings, set in code, because a confused agent will happily go round for ever at HM Nayem's expense. When a limit is reached, the agent stops and says exactly where it got to.
Watching, and stepping in
The model's reasoning and each tool request appear on HM Nayem's screen as they are produced. That is not decoration. It lets him stop the agent the moment it heads the wrong way. One key interrupts. The harness cancels the tool that is running, records exactly what was and was not done, and waits. HM Nayem types a correction, and it joins the conversation like any other message. The agent carries on from there with nothing lost.
Running with nobody there
The same harness can run without a person at all. When a bug in the tracker is given a certain label, a machine in the cloud starts the agent on it, in a fresh sandbox. With nobody to ask, the rules tighten: everything in "ask first" becomes "never". The agent cannot add its work to the shared code. It can only open a pull request, which waits for the reviewer built in an earlier story, and then for a person.
Version 9: test it on real bugs
To measure an agent you need tasks with a known right answer. The project's own history has them: thirty old bugs, each with the test that was written when it was fixed. For each one, the agent starts in a fresh sandbox, from the code as it was that day. It passes if the hidden test passes and no other test breaks.
What it still cannot do
HM Nayem starts using the agent every day, on its own branch of the code, with a person reading every change before it is added. It is useful and it is limited, and the limits are worth writing down while they are still fresh.
The finished agent, at real speed
The first task again, through everything we built. Notes in, search, read, two edits, the tests, one more edit, the tests again, and a report. Eleven rounds and fifty seconds. The model chose every step. The harness took every action, and decided which ones were allowed.
You built a coding agent, and saw that the model is the smallest part.
- The loop is six lines: call the model, run the tool it asks for, add the result, repeat.
- Give it search before reading, and small exact edits instead of whole files.
- Send errors back as results. A model that sees its mistake can correct it.
- A command runs in a kept-open shell inside three walls: files, network and limits.
- Permissions are written patterns. Take a command line apart and check every piece.
- Hooks run the team's rules every time. Checkpoints make every edit reversible.
- For big jobs, plan with read-only tools first and have a person approve the plan.
- Cut long results, summarise old rounds, and keep the plan outside the summary.
- Notes carry knowledge between sessions. MCP brings in other people's tools, to be treated as strangers.
- A helper is a new conversation with a role, fewer tools and a brief. Parallel helpers need separate copies.
- Set hard limits on rounds and money, and measure on real bugs with real tests.
Ideas in this story
- The agent loop: A model using tools round after round, deciding each next step from what just happened, until the job is done.
- The harness: The program around the model that runs the loop: it calls the model, runs the tools and keeps the record.
- Tool calling: The model cannot do anything itself. It asks your code to, and your code answers.
- The system prompt: Instructions the product's builders write, placed ahead of every user message.
- The sandbox: A sealed, throwaway computer for running code a model wrote.
- Code execution: The model writes a small program and a computer runs it, so exact work is done exactly.
- Least privilege: Giving an agent only the tools and access the current job needs, so a mistake has little to break.
- Asking first: Pausing before an action that sends something out or cannot be undone, and waiting for a person to say yes.
- Prompt injection: Text the model reads that carries instructions of its own, written by someone who wants to hijack it.
- Checkpoints: Saving an agent's progress after every step so a crash means resuming, not starting over.
- The context window: The fixed amount of text, counted in tokens, a model can take in at once.
- Compaction: Replacing a long record of work with a short summary so the agent can keep going.
- Prompt caching: The provider remembers the unchanging start of a prompt, so repeats of it cost about a tenth and arrive faster.
- Planning: The agent writes itself a to-do list, works through it, and rewrites it when reality disagrees.
- Long-term memory: Facts the app saves between conversations and puts back into the prompt when they are relevant.
- Model Context Protocol: A standard plug between assistants and tools, so each tool is connected once and works everywhere.
- Teams of agents: A lead agent hands parts of a job to helper agents, each with its own fresh window.
- Evals: A fixed set of test questions with known good answers, run after every change to see if things got better or worse.