Sentence Transformers

Embeddings without an API call

Sentence Transformers, also called SBERT, describes itself as "the go-to Python module for using and training state-of-the-art embedding and reranker models". It runs the model on your own hardware.

That is the reason to reach for it. Every provider page in this section (Cohere Embed, Gemini Embedding, OpenAI Embeddings API) sends your text somewhere. This one does not, which settles the question for data that cannot leave your network and removes a per-token cost from any corpus large enough for that to matter.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
embeddings = model.encode(["a sentence", "another sentence"])
scores = model.similarity(embeddings, embeddings)

Models load straight from the Hugging Face Hub, so the catalogue is the Hub's rather than a vendor's. Embedding models on Hugging Face covers choosing from it.

Four classes, for four jobs

The library is broader than one model type, and knowing which class you need saves building the wrong pipeline.

SentenceTransformer produces a dense embedding per input, and handles text, images, audio and video.

CrossEncoder scores a pair of texts rather than embedding them separately. Its methods are predict() and rank(). This is the reranker, and it is the one most RAG systems are missing: retrieve widely with a bi-encoder, then rerank the top results with a cross-encoder that reads query and document together. More accurate and far too slow to run over a corpus, which is why it goes second.

SparseEncoder produces sparse embeddings for SPLADE models, which behave more like learned keyword matching and catch the exact-term queries dense retrieval loses.

MultiVectorEncoder gives token-level embeddings for ColBERT-style retrieval.

A strong retrieval stack often uses three of these together, and having them in one library means one dependency rather than three.

Asymmetric search is explicit

Sparse and multi-vector encoders expose separate encode_query() and encode_document() methods.

That asymmetry is the same one Cohere Embed enforces with input_type: a question and the passage answering it are different kinds of text, and embedding them identically throws information away. Here it is two method names, which is harder to get wrong than a string parameter.

When you are using a plain SentenceTransformer, check whether the model expects a prefix on queries. Many do, and omitting it costs retrieval quality with no error.

Training is the capability the APIs do not have

SentenceTransformerTrainer, CrossEncoderTrainer and the equivalents support full fine-tuning, with configurable loss functions and datasets.

This is the thing a hosted embedding API cannot offer you. If your domain vocabulary is unusual, and retrieval keeps missing because "the model does not know our terms", fine-tuning an embedding model on your own pairs is a targeted fix rather than a general one. Fine-tuning is right that it shapes behaviour rather than adding knowledge, and for an embedding model the behaviour being shaped is exactly what you want: which things it considers similar.

Start with a reranker before you train anything. A CrossEncoder on top of existing retrieval is usually the larger improvement for the smaller effort.

What you take on

Running the model yourself means the same list as any local inference. Self-hosted models covers it, and the embedding-specific version is shorter: embedding models are small compared to generation models, so a modest GPU or even a CPU handles them, and the real work is throughput on an initial corpus embedding.

Two practical notes. Batch your encode() calls rather than looping, since the difference is large. And pin the model version, because re-embedding a corpus with a different model and querying with the old one gives you a corpus that is no longer searchable as one.

Further reading

• Sentence Transformers: the classes, encode(), similarity(), and the trainers.
• Embedding models: choosing between local and hosted.
• Cohere Embed: the hosted equivalent, and the same asymmetry.
• Self-hosted models: what running inference yourself involves.