The window fills up: AI Engineering, story 13
Forty rounds into a big job, the agent has read more than it can hold. See how it keeps going without losing the thread.
Long jobs overflow the model's view; the harness clears, takes notes and summarises to keep going.
The story, step by step
A much bigger job
HM Nayem hands the coding agent something far larger than one failing test. Sixty files need changing. This will take more than a hundred rounds of the loop, and a few hours.
Every round adds to the record
From the last story: the harness keeps a record of every request and every result, and sends all of it to the model each round. After five rounds it is still small. Watch the yellow part.
Tool results are the heavy part
The model's own requests are tiny. What it gets back is not. Reading one file puts the whole file into the record. Running the tests puts in every line of output. And all of it is resent every round after.
Round 40
Forty rounds in, the window is nearly full, and almost all of it is old tool results. Each round now costs far more than the first did, because the model is charged for reading the entire record again.
It gets worse before it breaks
Long before the hard limit, the work gets sloppier. With this much to look through, the model starts missing things it saw hours ago. Here it reads a file it already read in round six. A crowded window is a distracted one.
Fix one: clear what is stale
Most old tool results are no longer needed. A file read thirty rounds ago has been edited since. So the harness swaps each old result for a one-line stub: “file was read here”. The record of what happened stays. The bulk goes.
Fix two: write things down
A person on a long job keeps notes. So does a good agent. It writes progress and decisions to a file. Notes live outside the window, so they cost nothing until the moment they are needed.
Fix three: summarise and start fresh
Eventually the window fills again. Now the harness asks the model to summarise everything so far, and replaces the whole long record with that summary. This is called compaction. The agent continues with a nearly empty window and the thread intact.
What the summary loses
Compaction is not free. A summary keeps the goal, the decisions and what is left to do. It drops the raw detail. If the agent later needs an exact line from a file, it simply reads that file again.
Fetch when needed, not before
One more habit ties these together. The agent never loads all sixty files at the start “just in case”. That would not even fit. It reads each file at the moment it needs it, and lets it go afterwards.
The job finishes
A hundred and twelve rounds later, it is done. The model never saw more than a slice of the job at once. The harness chose that slice, every round. This craft has a name: context engineering.
The harness decides what the model sees.
- An agent's record grows every round, and tool results are most of it.
- A crowded window costs more and makes the model miss things.
- Stale tool results can be cleared, leaving a short stub.
- Notes in a file keep progress safe outside the window.
- Compaction replaces the long record with a summary so the work can go on.
Ideas in this story
- Context engineering: Deciding what goes into the model's window at each step, and what stays out.
- Compaction: Replacing a long record of work with a short summary so the agent can keep going.