What are Embeddings?

A list of numbers that stands in for meaning

An embedding is, in OpenAI's words, "a vector (list) of floating point numbers". Text goes in, a fixed-length list of numbers comes out, and the distance between two of those lists measures how related the two texts are.

That is the whole mechanism. What makes it useful is the property the numbers have: texts about the same thing land near each other, even when they share no words. "My package never showed up" and "the delivery is missing" are neighbours. "Bank account" and "river bank" are not.

Why this is different from search you already know

Keyword search matches the characters you typed. It has no opinion about whether "laptop" and "notebook computer" are the same thing, because as strings they are not.

An embedding moves the comparison from spelling to position. Once every document is a point, "find documents like this one" becomes "find the nearest points", which is arithmetic rather than string matching.

The cost of that move is that the numbers mean nothing on their own. You cannot read a vector and tell what it is about, you cannot explain to a user why one result ranked above another, and you cannot grep it.

The distance, and the one that is recommended

Two vectors are compared with a distance or a similarity measure. OpenAI recommends cosine similarity, and adds a detail worth keeping:

Cosine similarity can be computed slightly faster using just a dot product

because their embeddings are normalized to unit length. The same normalization means "Cosine similarity and Euclidean distance will result in the identical rankings."

So with unit-length vectors the choice of measure does not change the order of results. With vectors that are not normalized it does, which is the failure behind long documents dominating every result list. Recommendation systems covers that case. Whether a model returns unit-length vectors is a property of the model, so check before assuming.

Five things the same mechanism does

Distance between vectors answers more than one question, and the operations differ only in how you read the result:

OpenAI lists these same uses. It is worth noticing before you build infrastructure for the second one: a system that already embeds your catalogue for search is most of a recommender, which is the argument recommendation systems makes in full.

What embeddings do not do

They do not understand your domain. An embedding model learned from general text. If "P1" means something specific at your company, the vector does not know it.

They do not do exact match. Asking for order ORD-4417 by similarity is asking the wrong question; the nearest neighbours of an order number are other order numbers. Semantic search covers why keyword search still belongs in the system.

They are not comparable across models. Two vectors from two different models are lists of numbers in unrelated spaces, and the distance between them is meaningless. This is why changing embedding model means re-embedding everything, and why embedding models treats that as the real cost of the choice.

They are not reversible in a useful way, and not safe either. You cannot read the original text out of a vector, which is not the same as the vector being anonymous. Treat embeddings of sensitive text as sensitive.

Where this goes next

Embedding one string is an API call. Everything after that is the engineering: which model, what you embed, where you put the vectors, and how you search them.

The order those decisions arrive in is roughly embedding models, then semantic search, then a store that can do nearest-neighbour lookup at your scale. Purpose and functionality explains why an ordinary index cannot.

Further reading

• OpenAI, Embeddings: the definition, the recommended similarity measure, and the listed use cases.
• Embedding models: dimensions, input limits, and why the choice is hard to reverse.
• Semantic search: searching by meaning, and where keywords still win.
• OpenAI embeddings API: making the call.