Embeddings and Vector Similarity, Briefly

Turning meaning into numbers

Retrieval needs a way to ask "which of these chunks is about the same thing as this question?" without literally matching words. The answer is an embedding: a model that reads a piece of text and outputs a list of numbers, typically a few hundred to a few thousand of them, positioned so that texts with similar meaning end up close together in that numeric space, and unrelated texts end up far apart.

This is the same underlying idea as a language model itself: text is not compared letter by letter, it is compared as a point in a high-dimensional space shaped by meaning. "The engine won't start" and "the car doesn't turn on" land near each other even though they share almost no words, because an embedding model was trained on enough text to learn that they mean roughly the same thing.

Measuring "close"

Once text is a list of numbers, "similar" becomes a measurable distance. The most common measure in RAG is cosine similarity, which looks at the angle between two vectors rather than their raw distance, so it is not thrown off by one piece of text simply being longer than another. Retrieval, at its core, is: embed the question, compute cosine similarity against every indexed chunk, and return the highest-scoring ones.

This is also exactly what makes retrieval fast at scale: rather than comparing a question to every document's raw text, specialized vector indexes (the subject of this site's separate pgvector course) can search millions of embeddings for the closest matches in milliseconds.