Semantic Search

Matching meaning instead of characters

Keyword search finds documents containing the words you typed. Semantic search embeds the query, embeds the documents, and returns the nearest ones.

The difference shows up on the queries where the user and the document chose different words for the same thing. "My package never showed up" retrieves a document titled "Delivery exceptions and missing parcels", which shares not one term with the query.

That is the capability, and it is a real one. What follows is the part that gets skipped: it is not strictly better than keyword search, and the cases where it loses are specific enough to plan around.

Where keyword search still wins

These are the failure shapes worth knowing. They follow from what an embedding is rather than from any vendor's documentation, so treat them as reasoning about the mechanism and confirm them on your own corpus.

Exact identifiers. ORD-4417, ERR_CONN_REFUSED, a part number, a function name. An embedding puts these near other identifiers, because that is what they look like. Keyword search finds the one you asked for.

Rare terms. A word the embedding model barely saw in training has a weakly placed vector. Your internal project name, a new library, a drug name. Keyword search does not care how rare a token is. Rarity is what makes a match informative, so it arguably does better.

Quoted phrases. When a user quotes, they mean those characters in that order. Similarity has no way to express "exactly this".

Negation and small operative words. "Invoices not yet paid" and "invoices already paid" are close in embedding space and opposite in meaning. Short function words carry little weight in a vector averaged over a passage.

The pattern: semantic search is strong when the query is a description and weak when the query is a token.

Hybrid, which is what most production systems run

Because each method fails where the other works, the usual answer is to run both and merge. Weaviate's hybrid search "combines the results of a vector search and a keyword (BM25F) search by fusing the two result sets."

Two knobs matter. Alpha weights the halves: alpha = 0 is pure keyword, alpha = 1 is pure vector, and the default depends on the client library, so check it rather than assume a midpoint. Fusion method decides how the two result lists become one. Weaviate offers Relative Score Fusion, which uses the actual similarity scores and has been the default since v1.24, and Ranked Fusion, which uses positions.

The distinction is not academic. Ranked fusion treats the top keyword result and the top vector result as equally good. Relative score fusion can tell that one of them scored far higher than the other, which is usually the information you want.

What semantic search cannot fix

Bad chunks. The unit you embed is the unit you retrieve. A chunk that splits mid-explanation returns half an answer no matter how good the match. That is a chunking decision, covered in chunking strategies that survive production.

Missing content. Retrieval finds what is there. A question the corpus cannot answer returns the closest thing, confidently, which is why refusal has to be handled in the generation step rather than hoped for.

Ranking you cannot explain. When a result is wrong, "the vectors were close" is the only account available. Keeping the keyword scores alongside gives you something to debug with.

Getting an honest read on whether it is working

Write the queries before you build the index. Twenty real ones, with the document that should come back for each.

Then run three configurations over the same set: keyword only, vector only, and hybrid at a couple of alphas. Measure how often the right document appears at all, and how often it appears in the top three. The comparison is the point; a single number for the hybrid system tells you nothing about whether the vector half is earning its cost.

Include queries that are pure identifiers, and queries the corpus cannot answer. Those two categories are where the system embarrasses you in production, and they are the ones a happy-path test set never contains.

Further reading

• Weaviate, Hybrid search: BM25F fusion, the alpha parameter, and the two fusion methods.
• What are embeddings?: the vectors this searches over.
• Performing similarity search: top-k, filter ordering, and what a store does with the query.
• Retrieval process: where search sits in a full pipeline.