StoryboardAI Engineering

Embeddings

Turning text into a list of numbers so that similar meanings end up close together.

What are embeddings?

An embedding model reads a piece of text and returns a long list of numbers. It is a different kind of model from the one that writes replies: it produces no words, only this list.

The numbers work like a position on a map with hundreds of directions. Texts that mean similar things get positions near each other, even when they share no words. “Machine” and “front-loading washer” land side by side.

That turns search into geometry. Give the question a position with the same model, then look for the nearest stored positions. The store that does this quickly over millions of items is called a vector database.

An embedding places a text on a map of meaning.Step 1: A piece of text. Step 2: An embedding model reads it. Step 3: It returns a position. Step 4: Similar texts sit nearby.1A piece of text2An embedding modelreads it3It returns aposition4Similar texts sitnearby
An embedding places a text on a map of meaning.

Why do embeddings matter?

Searching by exact words fails whenever a question is phrased differently from the answer. Embeddings let software compare texts by what they mean. They are what makes it possible to find the paragraph about "sending something back" when the person asked about "returns".

How do embeddings work?

  1. A piece of text is sent to an embedding model, a different kind of model from the one that writes.
  2. It returns a long list of numbers, often around fifteen hundred of them.
  3. Think of the list as a position on a map. Texts with similar meaning land close together.
  4. To search, the question is turned into a position too, and the nearest stored positions are returned.

When should you use embeddings?

Use it when

  • Finding text by meaning
  • Grouping similar items, or spotting near-duplicates
  • Choosing the most relevant examples or tools for a request

Think twice when

  • The search is for an exact code, name or number. Meaning does not help there
  • You need to know why two things matched. The numbers cannot be read

Where do embeddings sit in a real system?

Embeddings are made ahead of time for every stored piece, and at question time for the question. They live in an index built to find the nearest positions quickly among millions.

What goes wrong with embeddings?

  • Comparing positions made by two different embedding models. They are not on the same map.
  • Forgetting to redo them when the text changes.
  • Expecting them to match rare codes or names, which carry no meaning to the model.

What does embeddings look like in practice?

The question "Can I wash my down jacket in a machine?" is nearest to a stored piece that says "Use a front-loading washer, cold, on a gentle cycle", though the two share almost no words. A piece about boot sizes is far away on the map.

Watch it happen

These stories show embeddings at work, one step at a time.