StoryboardAI Engineering

Compaction

Replacing a long record of work with a short summary so the agent can keep going.

What is compaction?

When an agent's window is nearly full, the harness can ask the model to summarise everything so far: the goal, what has been done, what was decided, what comes next. The long record is then replaced by that summary.

The agent carries on with a nearly empty window and the thread of the job intact. On very long jobs this happens several times.

Something is always lost: exact wording, full file contents, old output. A good summary keeps decisions and drops raw detail, and the agent can always read a file again if it turns out to need it.

Compaction replaces old detail with a short summary.Before: Old file contents and outputs; The last ten rounds; Nearly full. After: A short summary of early rounds; The last ten rounds, unchanged; Plan and notes kept separately.BeforeOld file contents and outputsThe last ten roundsNearly fullAfterA short summary of early roundsThe last ten rounds, unchangedPlan and notes kept separately
Compaction replaces old detail with a short summary.

Why does compaction matter?

A long conversation or agent run grows with every turn, and the whole of it is sent on each call. Sooner or later it will not fit, and long before that it becomes slow, costly and unfocused. Compaction is how a long job keeps going: older material is replaced by something much shorter that keeps what matters.

How does compaction work?

  1. The harness watches the size of the conversation against the model's limit.
  2. First it replaces old tool results with one-line stubs noting what happened.
  3. If that is not enough, it asks the model to summarise the early rounds.
  4. The summary replaces those rounds. The most recent rounds are kept word for word.
  5. The task, the plan and any standing notes are kept outside the part that is summarised.

When should you use compaction?

Use it when

  • A conversation or agent run is nearing the window's limit
  • Old tool outputs are no longer needed in full
  • Costs are rising with every turn

Think twice when

  • The conversation is short
  • Exact earlier wording will be needed again. Save it somewhere it can be fetched

Where does compaction sit in a real system?

It is done by the context manager in the harness, between rounds. It is triggered by size, not by the model.

What goes wrong with compaction?

  • Summarising the plan away, so the agent forgets what is left to do.
  • Compacting so often that detail needed two steps ago is gone.
  • Assuming a summary keeps everything. It always loses something.

What does compaction look like in practice?

After 60 rounds an agent's conversation holds 184,000 tokens, mostly old file contents. Rounds 1 to 50 are replaced by a 3,000-token summary. The last ten rounds stay as they were. The window drops to 37,000 tokens and the job carries on.

Watch it happen

These stories show compaction at work, one step at a time.