Embedding Models

The choice you make once

Picking an embedding model looks like picking any other API. It is not, because of one property: vectors from different models are not comparable.

Change the model and every stored vector becomes meaningless, because the new query vector lives in a different space. So the decision comes with a migration attached from the day you make it, and the migration is proportional to your corpus rather than to your code.

That is the frame for everything below. The specifications matter less than they appear, and the ones that lock you in matter more.

Dimensions

A model returns a fixed number of floats. On OpenAI's current line, checked 2026-09-25:

More dimensions can carry more distinction and cost more of everything: storage, memory, and time per comparison. A 3,072-dimension index is twice the bytes of a 1,536 one for the same documents.

The v3 models let you ask for fewer. The dimensions parameter shortens the vector. The documentation describes it as removing "some numbers from the end of the sequence" without losing the semantic properties, because the models are trained so the leading dimensions carry most of the signal.

One caveat with teeth: if you shorten a vector yourself rather than through the parameter, you have to re-normalize it. The documentation says manual reduction requires L2 normalization. Skip it and search gets quietly worse while nothing errors. Jina embeddings covers the same trap on another provider, which suggests it is the property of the technique rather than of one vendor.

Input limit

All three OpenAI models accept 8,192 tokens. That is the ceiling on one embedding call, and text beyond it is not embedded, so a chunking strategy that produces a 10,000-token chunk silently loses the tail unless you check.

The limit is also not the recommendation. A single vector summarising 8,000 tokens across six subjects is a blunt average that ranks moderately for everything. The useful chunk size is a retrieval decision, and chunking strategies that survive production makes it on its own terms.

What decides it

Given the lock-in, the specifications are rarely the deciding factor. Four things usually are.

Where it runs. A hosted API means your documents leave your network on ingest and your queries leave on every search. A local model means neither, and means you operate it. This is often the whole decision and it is not on any benchmark.

Cost at your volume. Embedding is charged per token and you pay it twice: once for the corpus, then again for every query, forever. A corpus you re-embed monthly has a different profile from one embedded once.

Whether it handles your text. Benchmarks are run on general prose. If your corpus is code, or legal text, or a language other than English, the ranking that matters is the one on your own documents. Two hundred labelled queries from your own traffic will tell you more than any leaderboard.

Whether you can leave. Ask what re-embedding would cost in money and in hours before you commit, not after. That number is the real price of the decision.

Mixing is the one thing you cannot do

Everything in an index must come from the same model, at the same dimension, with the same normalization and the same task convention where the model has one.

The failure is worth describing because it is so quiet. Embed half your corpus with one model and half with another and nothing errors: both are float arrays of plausible length, the store accepts them, and the search returns results. The results are wrong, ranked by distances computed between points in unrelated spaces. Nothing in the pipeline can detect it.

So record the model and its settings alongside the vectors. When you eventually migrate, that record is what tells you which rows still need doing.

Further reading

• OpenAI, Embeddings: the model line, dimensions, input limits, and the dimensions parameter.
• What are embeddings?: what the vectors are and what the distance means.
• Jina embeddings: long inputs, task types, and the same truncation caution.
• Embedding models on Hugging Face: running one yourself, and the re-embedding cost in full.