Types of Models

Five kinds, doing different jobs

"Which model" usually means "which of the big chat models", and that is one row of a larger table. Choosing the right model covers comparing within a class. This page is the classes, because reaching for the wrong kind is a more expensive mistake than picking the wrong member of the right one.

Base models

A base model predicts the next token and nothing else. It has not been taught to follow instructions, answer questions or stop. Pre-trained models covers why a raw model would not answer your question: given one, it continues it.

You will rarely call one. They matter because they are the starting point for everything below, and because a Hugging Face search will return them alongside instruction-tuned versions with names that differ by one suffix.

Instruction-tuned models

The default assumption behind most application code. Post-training teaches the model to follow an instruction, hold a conversation, and stop when finished.

Everything in prompt engineering assumes one of these. When a model ignores your instructions entirely rather than following them badly, check you did not download the base variant.

Reasoning models

These generate intermediate reasoning before answering, and the difference is billed rather than cosmetic. OpenAI's GPT and o-series makes the point that reasoning tokens are billed, invisible, and take up the window.

On some providers the distinction has collapsed into a setting rather than a model. Anthropic Claude's current lineup has adaptive thinking always on, steered by an effort parameter, so "is this a reasoning model" is no longer always a model choice.

Reach for one when the task has steps that can go wrong in the middle: multi-constraint problems, planning, debugging. Reach elsewhere for extraction, classification and formatting, where you are paying for deliberation over a question with no middle.

Embedding models

These return a vector rather than text, and they are a different tool rather than a smaller one. What are embeddings covers what the vector means.

Two properties make this choice unlike the others. Embedding models on Hugging Face notes it is stickier: changing one means re-embedding your whole corpus. And the query and the document often need different treatment, which Cohere Embed enforces with a required parameter.

Rerankers sit next to these: they score a query against a document rather than embedding either alone, which is what Sentence Transformers exposes as a cross-encoder.

Multimodal models

These take more than text as input, and sometimes produce more than text as output. Multimodal AI covers how media reaches a request and what it costs.

Two things to separate. A model that reads images is not necessarily one that generates them, and the two capabilities come from different models on most providers. And a multimodal input model still bills the media as tokens, which is the arithmetic image understanding covers.

Picking the kind before the member

The failure this taxonomy prevents is using a general chat model for a job a specialised one does better and cheaper.

Classification with an embedding model and a set of class centroids beats prompting a large model, as data classification describes. Reranking search results with a cross-encoder beats asking a chat model to sort them. Moderation with a purpose-built classifier beats asking politely, which is the argument in content moderation APIs.

So ask what shape the output is before asking which model is best. A vector, a score, a label and a paragraph are four different questions, and only one of them wants a chat model.

Further reading

• Choosing the right model: the comparison, once you know the class.
• Pre-trained models: what the base model is and why it will not answer you.
• What are embeddings: the kind that returns a vector.
• Multimodal AI: the kind that takes more than text.