Embeddings and Semantic Similarity

Turn a sentence into coordinates; similar meanings land close together. The whole retrieval stack is built on this.

Knowledge bases, semantic search, and related-content recommendations all run on the same thing: turning text into coordinates, then comparing distances. This section explains what those coordinates actually are.

Signs you'll recognize:

• The knowledge base answers "how do I get a refund" fine, but when a user says "I don't want it anymore" it finds nothing.
• An engineer says "just add a vector store," and you're not sure how much work that sentence actually covers.
• Search results all look relevant, but the one you actually need isn't there.

What it solves

Start with the old approach's flaw. Traditional keyword search is literal matching: you search "refund" and the system looks for documents containing the word "refund."

The problem is that one meaning has many phrasings. "Refund," "money back," "I don't want it anymore," "request a return"—no literal overlap, but the same intent. Keyword search misses all of them.

Vectors fix exactly this. They don't look at wording; they look at meaning. "I don't want it anymore" and "how do I apply for a refund" share no words at all, yet once turned into coordinates the two points sit close together.

Think of semantic space as a map arranged by meaning: refund discussions cluster in one neighborhood, shipping in another, weather far away. Retrieval means placing the user's question on the same map and seeing which points are nearest.

How text becomes coordinates

The job belongs to a dedicated model called an embedding model. It's a different kind of thing from the text models you already know: text models take words in and produce words out; an embedding model takes words in and produces a string of numbers out.

That string usually has hundreds to thousands of values—hundreds to thousands of dimensions. Diagrams draw it in two dimensions just so it's visible, but the intuition "closer distance equals closer meaning" holds in high-dimensional space too.

These coordinates aren't hand-designed. The model learns them from huge amounts of text: words and sentences that appear in similar contexts get pulled closer together. So they reflect how language is used in the corpus, not any objective truth—and that gap becomes a problem later. word-embedding-wiki

The usual distance measure is cosine similarity. You don't need its math; just know it outputs a number between 0 and 1, where higher means more similar. OpenAI's embedding documentation recommends cosine similarity for ranking search results. openai-embeddings

The good news: this layer is cheap. Embedding models are the cheapest of the model types, usually one or two orders of magnitude below text models. So vectorizing your entire document set is barely a cost concern.

What it means for your product

Three things are worth knowing.

One: it's a one-time preprocessing step. Your documents get split into chunks, each chunk gets embedded, and the vectors go into a vector database. This happens offline. At query time you only embed the question once and compare distances, which is why queries are fast.

Two: updating a document means recomputing its vector. Change one doc and the embedding must be regenerated. This is easy to forget, and the result is a knowledge base still answering with three-month-old content. The update mechanism has to be designed in from the start.

Three: switching embedding models means recomputing everything. Different embedding models produce incompatible coordinate systems. Mixing them is like measuring distances on two different maps—the result is meaningless. Choose carefully; a switch is expensive.

Limits you should know

Vectors are useful, but they carry built-in limitations. Knowing them in advance avoids a lot of pain.

One: it measures "similar," not "correct." A passage that looks like the question may not contain the answer, and a passage that answers it may not look like the question. This is the most important caveat, covered in detail in the next article.

Two: it's insensitive to negation. "Refunds supported" and "refunds not supported" sit close together in vector space because their wording is almost identical. In compliance and policy knowledge bases, that's a real risk.

Three: exact matching is a weakness. Order numbers, product SKUs, statute numbers—vector retrieval is often worse than old-fashioned keyword search here. That's why production systems almost always combine both approaches.

Four: long text gets diluted. Compressing a very long passage into one coordinate averages all its topics together, and none stands out. So chunking strategy directly determines retrieval quality.