AI Engineering, story 13The window fills up

A much bigger job

1 of 11

The window fills up: AI Engineering, story 13

Forty rounds into a big job, the agent has read more than it can hold. See how it keeps going without losing the thread.

Long jobs overflow the model's view; the harness clears, takes notes and summarises to keep going.

The story, step by step

  1. A much bigger job

    HM Nayem hands the coding agent something far larger than one failing test. Sixty files need changing. This will take more than a hundred rounds of the loop, and a few hours.

  2. Every round adds to the record

    From the last story: the harness keeps a record of every request and every result, and sends all of it to the model each round. After five rounds it is still small. Watch the yellow part.

  3. Tool results are the heavy part

    The model's own requests are tiny. What it gets back is not. Reading one file puts the whole file into the record. Running the tests puts in every line of output. And all of it is resent every round after.

  4. Round 40

    Forty rounds in, the window is nearly full, and almost all of it is old tool results. Each round now costs far more than the first did, because the model is charged for reading the entire record again.

  5. It gets worse before it breaks

    Long before the hard limit, the work gets sloppier. With this much to look through, the model starts missing things it saw hours ago. Here it reads a file it already read in round six. A crowded window is a distracted one.

  6. Fix one: clear what is stale

    Most old tool results are no longer needed. A file read thirty rounds ago has been edited since. So the harness swaps each old result for a one-line stub: “file was read here”. The record of what happened stays. The bulk goes.

  7. Fix two: write things down

    A person on a long job keeps notes. So does a good agent. It writes progress and decisions to a file. Notes live outside the window, so they cost nothing until the moment they are needed.

  8. Fix three: summarise and start fresh

    Eventually the window fills again. Now the harness asks the model to summarise everything so far, and replaces the whole long record with that summary. This is called compaction. The agent continues with a nearly empty window and the thread intact.

  9. What the summary loses

    Compaction is not free. A summary keeps the goal, the decisions and what is left to do. It drops the raw detail. If the agent later needs an exact line from a file, it simply reads that file again.

  10. Fetch when needed, not before

    One more habit ties these together. The agent never loads all sixty files at the start “just in case”. That would not even fit. It reads each file at the moment it needs it, and lets it go afterwards.

  11. The job finishes

    A hundred and twelve rounds later, it is done. The model never saw more than a slice of the job at once. The harness chose that slice, every round. This craft has a name: context engineering.

The harness decides what the model sees.

Ideas in this story