LanceDB

A retrieval library rather than a database

Most vector stores are services. You run a process, or you pay someone to run it, and your application talks to it over a network.

LanceDB describes itself as "a developer-friendly OSS embedded retrieval library for multimodal AI", built "on top of the Lance columnar format". The word that changes the proposition is embedded: you import it, and there is no server.

That puts it in the same argument as Supabase Vector, from the opposite direction. One says use the database you already run. This one says do not run a database.

What embedded means in practice

No process to supervise, no port to secure, no deployment step, and no network hop between your code and your data. A test suite can build an index and throw it away. A notebook can open the same data your application uses.

It also claims to run "locally or in your cloud. No vendor lock-in", with a managed offering alongside for teams that want "no servers to manage".

The cost of embedding is the usual one. Your search runs in your process, competing for the same CPU and memory as everything else it does, and scaling search means scaling whatever your application runs on. That is fine for a single service and awkward when several services want the same index.

Vectors, metadata and multimodal data in one table

The second distinguishing claim: it stores "vectors, metadata and multimodal data (text, images, videos, point clouds, and more)" together.

For a text pipeline this is convenient. Your chunk text sits with its embedding, so a search returns something you can quote without a second lookup, which is the practice storing in a vector database recommends anyway.

For a pipeline that is not text, it is the point. If you are searching over images or video, the assets are large and the usual arrangement splits them from their embeddings: vectors in one store, files in object storage, and identifiers gluing the two together. Keeping them in one table removes that seam and the class of bug where the two disagree.

The project claims the ability to "store, index, and search over petabytes of multimodal data and vectors", which is a scale claim I have not tested and am reporting as theirs.

The Lance columnar format underneath

Lance is a columnar format, which means values from the same field are stored together rather than row by row.

For analytical work that layout is why you can read one column of a large table without reading the rest. For retrieval it means the vector column, the metadata you filter on, and the payload you return are separable at read time, so a query does not have to pull whole records to evaluate a filter.

That is the general property of columnar layouts rather than measured behavior in this system. The format is the project's stated foundation, not a benchmark run here.

When an embedded store fits, and when it does not

It fits when your corpus is multimodal, when you want retrieval inside a process rather than beside it, when a local development story matters, or when adding another service is the thing you are trying to avoid.

It fits less when several services need to query the same index, when search has to scale independently of your application, or when you need a store your operations team already knows.

Between this and pgvector, the question is what you already run. A team on Postgres gets more from a vector column than from a new library. A team with no database, or with a corpus that is mostly images, gets more from this.

What the project does not claim

The seeded description of this tool on this site said it offers on-disk storage that scales past memory. The project's own README does not present on-disk indexing as a distinct feature, and does not claim scaling beyond available memory as a capability, so neither is stated here.

If the memory ceiling of your current store is the problem you are solving, verify that specific behavior against current documentation rather than taking it from a summary, including this one.

Further reading

• LanceDB, the project repository: the claims quoted above, and the supported languages.
• Supabase Vector: the other answer to not running a vector service.
• Purpose and functionality: what this category of store does at all.