Fix the failing test: AI Engineering, story 12
HM Nayem tells a coding agent one sentence: the checkout test is failing, fix it. Watch it look, read, change, check and try again, on its own.
An agent is a model using tools in a loop with a plan it keeps rewriting; the harness is the program running that loop.
The story, step by step
One sentence, a whole job
HM Nayem is building a website for a hiking club, and one of its tests has gone red. This time the message is not a question. It is a job. Nobody knows yet how many steps it will take, not even the agent.
A new kind of program
In earlier stories the shop's server called the model once or twice and stopped. This program is different. It keeps going back to the model, round after round, until the job is done. It is called a harness. A model running inside one is called an agent.
What the agent can do
Like the order lookup, each tool is an ordinary function with a short description. Three are enough for this job: read a file, change a file, and run the tests.
First, a plan
Before touching anything, the model writes itself a short to-do list. Nobody told it how to solve this. The list is its own guess at the steps, and it keeps a long job on track.
Round 1: look
Round one. The model asks for the tests to be run. The harness runs them and sends the result back. Now the agent knows what is failing: a total that should be 108 came out as 100.
Round 2: read
Round two. Nobody told the agent where the bug is. It follows the evidence: a wrong total, so read the code that works out totals. And there it is. A tax rate is passed in, and never used.
Round 3: act
Round three is the first one that changes anything. The model asks for one line to be edited so that tax is added. The harness makes the change in the real file.
Round 4: check, and a surprise
The agent does not assume its fix worked. It runs the tests again. They still fail, by a hair: a rounding error far past the decimal point. No plan written in advance could have predicted this.
The plan changes
So the agent rewrites its own list. This is the difference between an agent and a script. A script follows fixed steps. An agent chooses the next step from what just happened.
Rounds 5 and 6
One more edit, to round the total to two decimal places. One more test run. This time every test passes.
Knowing it is done
How does the loop end? Each round, the model either asks for a tool or it does not. This time it replies with plain words and no tool request. The harness takes that as “finished” and stops calling it.
The report
HM Nayem typed one sentence and gets one back. In between, the agent took six steps that nobody listed for it, including one it had to invent halfway through.
What held it together
Remember, a model has no memory. So how did round six know about round one? The harness kept the full record and sent all of it again each round. That growing record is the agent's whole working memory. It is also the subject of the next story.
Now at real speed
Look, read, change, check, change, check, done. An agent is nothing more than a model, some tools and a loop. What makes it feel clever is that each step is chosen from the result of the last.
An agent is a model, tools and a loop.
- The harness is the program that calls the model round after round and runs its tools.
- Each round, the model chooses the next step from the result of the last one.
- It writes a to-do list, and rewrites it when something unexpected happens.
- The loop ends when the model replies without asking for a tool.
- The harness keeps the full record and resends it every round.
Ideas in this story
- The agent loop: A model using tools round after round, deciding each next step from what just happened, until the job is done.
- The harness: The program around the model that runs the loop: it calls the model, runs the tools and keeps the record.
- Planning: The agent writes itself a to-do list, works through it, and rewrites it when reality disagrees.