Data Classification

Classify by distance, not by prompting

The instinct for labelling text is to ask a model. For a large volume of items and a fixed set of classes, there is a cheaper method that is often as accurate: embed everything once, and compare each item to a representative point per class.

The representative point is a centroid, the average of the embeddings of labelled examples for that class. Classifying an item means embedding it and finding the nearest centroid, which is the same nearest-neighbour operation performing similarity search describes.

The arithmetic is what makes this worth knowing. Embedding is one cheap call per item. Prompting a chat model is one expensive call per item, forever.

The method

Four steps.

Collect labelled examples, a few dozen per class. They need to span the class rather than sit in the middle of it, because an average of five near-identical examples describes a narrower region than the class occupies.

Embed them, using the same model you will use for incoming items. Any mismatch invalidates the comparison, which is the versioning discipline embedding models insists on.

Average per class to get a centroid. Normalize afterwards if you intend to use dot product, for the reason performing similarity search gives about metric choice.

Compare and label. Embed the new item, measure against each centroid, take the nearest.

If your embedding provider distinguishes query from document text, use it here. Cohere Embed exposes a classification value of input_type for exactly this case, and using the retrieval values instead leaves quality on the table.

The distance is a confidence signal

The useful property, and the reason this beats a prompt in one specific way: you get a number.

A model asked to pick a category returns one, with no honest indication of how close the call was. Centroid distance gives you the margin directly. Two things follow.

Set a threshold for "none of these". An item far from every centroid is probably not in your taxonomy, and saying so beats forcing it into the nearest class. A prompt-based classifier will cheerfully pick one.

Route the ambiguous cases. Where the top two centroids are close, send the item to a model or a person. You then pay the expensive path for the small fraction that needs it, which is the arrangement that makes the economics work.

Where it beats a model, and where it does not

Beats a model when classes are semantically distinct, volume is high, you have labelled examples, and the taxonomy is stable. Topic routing, intent detection, tagging a document corpus, deduplicating support tickets.

Loses to a model when the distinction requires reasoning rather than similarity. "Is this contract clause unusual" is not a distance question. It also loses when classes overlap in wording, or when you have no labelled examples at all, where a prompt with a few-shot list is a faster start.

Types of models makes the general version of this argument: ask what shape the answer is before choosing the model. A label from a fixed set is a shape an embedding model serves well.

Measuring it

This is a classification problem with a right answer, so it gets the cheap tier of evaluation. Deterministic evals covers computed checks, and evaluation metrics is right that precision and recall per class beat a single accuracy figure: one class failing badly disappears into an average.

Hold out labelled examples rather than scoring on the ones that formed your centroids. Scoring against your own training points measures nothing.

When a class performs badly, the usual cause is that its examples were too similar to each other. Add examples from the edges before changing anything else.

Further reading

• What are embeddings: what the vectors mean.
• Performing similarity search: metrics, and why normalization decides which one.
• Embedding models: choosing one, and keeping it consistent.
• Evaluation metrics: per-class precision and recall.