OpenAI Embeddings API

Making an embeddings call and reading the result

You send text and get back a list of floating-point numbers with a fixed length. That list is the embedding, and its usefulness comes from position: texts about similar things land near each other.

Billing works the way the rest of the API does, per input token. Both current v3 models accept up to 8192 tokens per input, which is long enough for most chunks and short enough that a whole document needs splitting first.

Choosing between the embedding models

OpenAI documents three: text-embedding-3-small at 1536 dimensions, text-embedding-3-large at 3072, and text-embedding-ada-002, marked legacy.

Reach for small first. It costs less per token, its vectors take half the storage of large, and searching them is faster. Move up when you have measured retrieval quality on your own corpus and found it wanting, which is a different thing from assuming the bigger number wins.

Model names move. Check the current roster and prices in the guide rather than trusting a list in an article, this one included.

Shortening vectors with the dimensions parameter

A dimensions parameter returns a shorter vector. The guide describes the mechanism as shortening embeddings, meaning you "remove some numbers from the end of the sequence" without the embedding "losing its concept-representing properties."

The trade is quantified in the documentation, and the number is more favorable than most people expect. text-embedding-3-large cut to 256 dimensions scores 64.6% on the MTEB benchmark. The full 1536-dimension ada-002 scores 61.0%. A vector six times shorter beats the older model outright.

That changes how you should think about storage. A quarter-length vector is a quarter of the memory in your index and a proportional cut in search time, for a quality cost you can measure rather than guess at.

Truncate at creation, not afterward

If you shorten a vector yourself by slicing the array, you have to L2-normalize the result. Using the dimensions parameter is the documented suggested approach, and it handles this for you.

Skip the normalization and nothing fails. Your index accepts the vectors, your queries return results, and the results are quietly worse. This is the kind of bug that survives to production because it produces no error, only a slow retrieval quality problem nobody can trace.

Embedding a corpus in batches

The API accepts an array of inputs in one request. Use it.

Embedding a corpus one string at a time means paying request overhead thousands of times for work the API will do in one call. The token cost is identical; what you save is round trips, and on a large corpus that is the difference between a job you watch and a job you schedule.

Keep your own identifier alongside each input so you can match vectors back to chunks. The API returns embeddings in the order you sent them, and relying on that order without a key is a bug waiting for the first retry.

The constraint that outlives any model name

A query and the corpus it searches must be embedded by the same model. Vectors from two different models occupy different spaces, and comparing across them returns numbers that mean nothing.

So changing your embedding model invalidates every vector you have stored. This is not a migration you can do incrementally, and it is the main reason to plan for re-indexing before you need it. The write-side article covers how: storing in a vector database.

Further reading

• OpenAI, Embeddings: the model family, the dimensions parameter, and the MTEB comparison above.
• What are embeddings: the concept, if the first section assumed too much.
• The MTEB leaderboard: how these models compare with open alternatives.