Chroma

Three clients, and what each does with your data

Chroma describes itself as "the open-source data infrastructure for AI", covering embedding storage, vector search and metadata filtering. What distinguishes it in practice is how little you have to stand up to start.

There are three ways to connect, and the choice is about where the data lives.

Ephemeral. Everything in memory, gone when the process exits. Useful for tests and for a notebook you are going to throw away.

Persistent. Runs inside your application process and writes to a directory you name. Data is saved automatically and loaded again on start. Nothing to deploy, no port, no second process.

Client-server. Your application talks over HTTP to a Chroma server running separately, which is what you need once more than one process wants the same data. A managed cloud option exists alongside it.

The client type is the line you change. The collection API you write against is the same in each, and the rest of this page turns on that property.

Client details here come from Chroma's community cookbook rather than the vendor's own reference, which did not enumerate them on the page read on 2026-09-16.

Why the embedded default is a reasonable place to start

Three vector stores are already covered here, each arguing a position. Supabase Vector says use the database you already run. LanceDB says do not run a database. Weaviate says run a service whose job is search.

Chroma's actual position on most projects is more mundane and worth naming: it is the one people reach for first, because the persistent client gets a working query in about four lines and no infrastructure decision.

That is a legitimate reason to choose it. Retrieval quality depends on chunking, the embedding model and your filters far more than on which store holds the vectors, and those are all things you learn by having something working. A store that lets you get there today and defers the operational question is a reasonable trade at the start of a project.

The questions that arrive when the prototype gets traffic

The trade comes due in a predictable order. These are the questions to answer against current documentation before the embedded setup carries real traffic, rather than after.

Who else needs this data? An in-process store is reachable by that process. A second web server, a background worker, or a scheduled job wanting the same collection is the point where client-server mode stops being optional.

What happens when two things write at once? Concurrency behaviour for the persistent client is the question I could not answer from the pages I read, and it is the one most likely to bite. Establish it before you have two writers, not after.

Where does the directory live? A local directory is a real dependency. On a container that is rebuilt on deploy, "persistent" means until the next release unless the path is on a volume that outlives it.

What is the backup? A directory of files needs a backup story of its own, which is the thing a managed database was quietly doing for you.

None of these are objections to the tool. They are the bill for the setup you skipped, and it is a smaller bill than most.

What moving costs

Two moves look similar and are not.

Moving from the embedded client to a Chroma server changes which client you construct and where the data sits. Your collection code does not change. That is a deployment task, and a small one.

Moving to a different vector store is a migration. You re-create the collections, re-load the vectors, and rewrite whatever filter syntax you used, because that part is not portable. If you also change embedding models on the way, you are re-embedding the corpus as well, which embedding models on Hugging Face covers.

That asymmetry is what to weigh when choosing. Starting embedded costs you a deployment step later. Starting on the wrong store costs a migration.

Further reading

• Chroma, Introduction: what it covers and the managed option.
• Chroma Cookbook, Clients: the client types, as community documentation.
• Purpose and functionality: what this category of store does.
• LanceDB: the other embedded argument, made from multimodal data.