AI Engineering, story 6Finding the right page

One question, two thousand pages

1 of 11

Finding the right page: AI Engineering, story 6

The shop has two thousand help pages and HM Nayem has one question. Watch the right three paragraphs get found, by meaning, in a blink.

Documents are cut up and found by meaning; the model answers only from those pieces and shows its sources.

The story, step by step

  1. One question, two thousand pages

    HM Nayem wants to know about washing a down jacket. The answer exists, somewhere in the shop's two thousand help pages. The server has about a tenth of a second to find it.

  2. Long before the question

    The work starts long before anyone asks. Every help page is cut into small pieces called chunks, a paragraph or two each. Small pieces are easier to find and cheaper to hand to the model.

  3. Each chunk gets a position

    Next, a second kind of model, an embedding model, turns each chunk into a list of numbers. Think of the numbers as a position on a map. Chunks that mean similar things land close together.

  4. What a position really is

    Here is one chunk and its position: a list of 1,536 numbers. No person can read them, and nobody needs to. All that matters is that two texts with similar meaning get similar numbers. The map on screen is that idea, flattened to two directions.

  5. Stored, ready to search

    All 8,400 positions are saved in a search index, also called a vector database. This preparation happens once, and again for any page that changes. From here on, every question can use it.

  6. The question gets a position too

    Now the question arrives. The server runs it through the same embedding model, so it lands on the same map, right beside the chunks about the same subject.

  7. The nearest chunks win

    Finding the answer is now just finding the nearest points. Notice that the question says “machine” and the best chunk says “front-loading washer”. No shared words. It matched on meaning.

  8. Build the prompt

    The server does what it did in the last story: it puts the text in the prompt and says to answer only from it. The difference is that these three chunks were chosen automatically, out of 8,400.

  9. An answer with sources

    The model answers from the chunks and points to where each fact came from. HM Nayem can check the source. So can the shop, when something looks wrong.

  10. It has a name

    This pattern is called retrieval-augmented generation, or RAG. Retrieve the right text, add it to the prompt, generate the answer. It is how most AI products know things their model was never taught.

  11. Now at real speed

    At real speed, the search is the fastest part. Two thousand pages, three paragraphs, one grounded answer, in about the time it took to read this sentence.

The model answers from text it was handed a moment ago.

Ideas in this story