AI Engineering, story 14It remembers me

Monday

1 of 12

It remembers me: AI Engineering, story 14

HM Nayem comes back a week later and the assistant still knows the right boot size. The model remembers nothing. So who does?

How an assistant recalls last week when the model itself remembers nothing.

The story, step by step

  1. Monday

    HM Nayem is back at the shop's chat, looking for boots, and mentions two things in passing: a size, and a rule about materials. The assistant uses both and finds a good pair.

  2. The chat ends

    Remember how a chatbot seems to remember: the server resends the whole conversation each time. That only lasts as long as the conversation does. When HM Nayem closes the chat, there is nothing left to resend.

  3. A week later

    HM Nayem opens a new chat and asks a question that only makes sense if the shop remembers. It does not. Nothing broke. This is simply what every chatbot does unless someone builds more.

  4. Decide what is worth keeping

    So the shop builds memory. First question: remember what? Not the whole chat. Most of it is small talk. When a conversation ends, the server asks the model to pick out the few facts that will still matter next time.

  5. Save it outside the model

    Those facts are saved in an ordinary database the shop owns, filed under the customer. The model is not changed or retrained in any way. Nothing has been taught to it.

  6. Next time, look first

    Now replay the following week. A new chat begins. Before calling the model, the server looks up what it has saved about this customer.

  7. Into the prompt

    The saved facts go into the prompt, between the shop's instructions and the new message. To the model, it is just more text to read. It cannot tell these lines came from last week.

  8. It remembers

    And there it is. The model did not remember anything. The shop's code remembered, and told it. From the phone, the two are impossible to tell apart.

  9. A year of memories

    After a year there are hundreds of saved facts, and most have nothing to do with boots. Sending all of them would waste the window. So the server searches its memories by meaning, the way it searched help pages, and sends only the closest few.

  10. Memories go out of date

    Then one day the size changes. If the server only ever adds, it now holds two sizes and will keep getting it wrong. Memory has to be edited, not just piled up. A stale memory is worse than none.

  11. Whose memory is it?

    One last thing. This is a file of personal facts about a real person. A good product lets that person see what is kept and delete any of it, and never saves the sensitive things at all.

  12. Now at real speed

    A new message arrives, the server looks up what it knows, adds it to the prompt, and the model answers like an old friend. Then anything new worth keeping is saved for next time.

The model forgets. The app remembers for it.

Ideas in this story