RAG with Dynamic Filters

When a filter stops being about relevance

Most writing about metadata filters treats them as tuning. Narrow to recent documents, prefer this category, boost that source. Get it wrong and results are worse.

There is a second use where the same mechanism carries a different weight. When each customer must only ever see their own documents, the filter is the only thing standing between a search and a data breach. Get it wrong and you return another company's contract.

Same API call, entirely different consequence for a bug. Everything below follows from taking the second case seriously.

A predicate and a partition are not the same guarantee

Vector stores offer two mechanisms that look interchangeable in the documentation.

A metadata filter is a condition applied to a query: return nearest neighbours where tenant_id equals this value. It is an argument you pass, and it is only applied when you pass it.

A namespace, or collection, or partition depending on the store, is a boundary the query runs inside. Pinecone's constraint, quoted in storing in a vector DB, is that all operations target one namespace. A query cannot reach outside it, because there is no expression for outside.

The difference is what happens when someone forgets. A forgotten filter returns everything. A forgotten namespace returns an error, or the wrong empty result, because the query has to name one.

For a tenancy rule, that asymmetry is the whole argument. You want the failure mode where a mistake returns nothing rather than the one where a mistake returns everyone's data.

This is reasoning from how the two mechanisms work rather than a rule any vendor publishes. Stores differ in what they offer and what it costs, so check yours. But when both are available and the rule is about who may see what, the partition is the safer default and the filter is the refinement inside it.

Where the filter runs, and what that changes

Performing similarity search covers the ordering question in full: a filter applied after the vector scan throws away results you already paid for and can return fewer than you asked for, while a filter applied first narrows what gets scanned.

For tenancy that matters twice over. The obvious way is performance, since scanning one customer's thousand documents beats scanning ten million and discarding. The less obvious way is correctness under a top-k limit. Ask for ten nearest neighbours and filter afterwards, and a customer whose documents are all outside the global top ten gets nothing back, while a customer with a large corpus gets a full page. The system appears to work better for bigger accounts, which is a confusing bug to chase.

Building the filter from identity rather than from input

Here is the failure worth designing against, and it is ordinary.

// The tenant is whatever the caller said it was.
const { query, tenantId } = await request.json()
const results = await index.query({ vector, filter: { tenant_id: tenantId } })

That is the same mistake as trusting a client-supplied user ID anywhere else, and it is easier to make here because the filter looks like search configuration rather than authorisation. It sits next to topK and includeMetadata, so it reads like tuning.

// The tenant comes from the session, and the caller cannot choose it.
const session = await getSession()
if (!session) return unauthorized()
const results = await index.query({ vector, filter: { tenant_id: session.tenantId } })

The rule: the scoping value is derived server-side from the authenticated request, never read from the body, the query string, or a header the client controls. If your retrieval function takes a tenant ID as a parameter, make it take a session instead, so there is no way to call it without one.

Testing that the boundary holds

Tenancy bugs do not show up in ordinary use, because in ordinary use everyone has their own data and sees it.

Two tests catch most of it, and both belong in the suite that runs on every change rather than in a one-off check.

Cross-tenant retrieval. Seed two tenants with documents. Query as tenant A with text that matches tenant B's document closely. Assert the result set is empty rather than merely not containing it, since an empty assertion fails loudly if the filter silently stops applying.

Filter-stripping. Call your retrieval path with the scoping value removed, and assert it errors rather than returning everything. If deleting one line from a query returns the whole corpus, that line is doing security work with no backstop.

Regression testing covers keeping those cases running as the retrieval pipeline changes, which is exactly when a filter quietly stops being applied.

Further reading

• Storing in a vector DB: namespaces as hard partitions, and the metadata constraints that come with them.
• Performing similarity search: filter ordering and what it costs.
• Weaviate: one store's filtering and index choices, including the per-tenant case.
• RAG use cases: whether retrieval is the right shape at all.