Performing Similarity Search

What a similarity query asks the store for

You have a query embedding and a store full of document embeddings. A similarity search finds the stored vectors closest to yours and returns them in order.

Three decisions sit inside that sentence: how you measure closeness, how many results you take, and what you exclude before measuring. Each one changes the answers you get back, and the third one changes them more than most people expect.

Choosing between cosine, dot product, and Euclidean distance

Cosine similarity measures the angle between two vectors and ignores their lengths. Two documents about the same subject point the same way whether one is a paragraph or a page.

Dot product multiplies the vectors element by element and sums the result. It responds to both direction and magnitude, so a longer vector scores higher for the same orientation.

Euclidean distance measures the straight-line gap between the two points.

The three are not as different as they look. Qdrant states that "Dot Product is equivalent to Cosine distance for normalized vectors," and once every vector has length 1, magnitude stops varying and only angle remains. Supabase adds the practical half: "dot product tends to be the fastest if your vectors are normalized."

So on normalized vectors, cosine and dot product rank results identically and dot product costs less. Pick the metric your embedding model was trained for, normalize, and then choose on speed.

Why normalization decides which metric to use

Most text embedding models return normalized vectors, or document that you should normalize before comparing. When you skip it, magnitude leaks into your ranking.

The symptom is specific: long documents start winning. Their vectors are longer, dot product rewards length, and your top results fill with the biggest chunks in the corpus regardless of subject. Switching to cosine masks it, since cosine ignores magnitude, but the underlying vectors are still not what your model expects.

If you shorten vectors yourself rather than asking the provider for fewer dimensions, normalize afterward. That is the same trap the OpenAI embeddings API article covers from the other side.

Picking k against the context budget downstream

Retrieval is not the last step. Whatever you return goes into a prompt, and the prompt has a budget.

Work backward. If your chunks run 500 tokens and you have 4000 tokens to spend on context, k above 8 sends material you cannot use. Raising k until results look good in a test harness produces a number that silently truncates later.

Retrieving more than you send is a real strategy, but only with something in between: a reranker that reorders candidates, or a filter that drops the ones below a score. Fetch 50 and keep 5 is deliberate. Fetch 50 and paste all of them is an accident.

Filtering before, during, or after the search

Here is the decision people get wrong, and it is not a subtle mistake.

You want the 10 closest documents belonging to one customer. There are two orders to do that in.

Filter after searching. Ask for the 10 nearest across the whole store, then drop the ones belonging to other customers. If your customer owns 1% of the corpus, you will usually get zero results. You asked for the global top 10 and then deleted most of it.

Filter during the search. The store only considers vectors matching the condition, so the 10 that come back are the 10 nearest that belong to your customer.

Qdrant applies filters during search for exactly this reason. Post-filtering does not shrink your results occasionally under bad luck; it shrinks them by construction, every time, in proportion to how selective your filter is.

Check what your store does. In pgvector the WHERE clause and the ordering are part of one statement, so the planner handles it. In a dedicated vector database, filtering is usually a parameter on the query rather than something you apply to the results, and passing it is the difference between a working search and an empty one.

Three reasons your results disappoint

When a search returns something unhelpful, work through these before touching the model.

The metric disagrees with the model. You are using dot product on unnormalized vectors, or a metric other than the one the model was trained for. Ranking degrades without anything failing.

The filter ate the candidates. You are post-filtering, or filtering on a field that is more selective than you assumed. Run the query without the filter and see whether the right document appears.

The chunk never held the answer. Retrieval cannot return what nobody stored. If the passage that answers the question is split across two chunks, neither scores well on its own. That is a chunking problem, and chunking strategies that survive production covers it.

Further reading

• Qdrant, Search: metrics, the normalized-vector equivalence, and filtering during search.
• Supabase, Vector columns: pgvector's operators.
• Storing in a vector database: the write side, including the metadata you filter on here.