Fine-tuning

What fine-tuning changes, and what it does not

Three ways to change what a model does for you, and they act on different things.

Prompting changes what you ask. Retrieval changes what the model is given at request time. Fine-tuning changes the weights, by training further on examples of the behavior you want.

Only the third modifies the model. That is why it feels like the powerful option, and why it is reached for in situations where it cannot help.

Behavior and format, not knowledge

Look at what a provider lists as fine-tuning's benefits. OpenAI's guide names consistently formatting responses a certain way, handling novel inputs, classification, and correcting instruction-following failures, plus fitting more examples than one request allows, cutting token costs, training on sensitive data, and getting a smaller model to do a narrow task well.

Notice what is absent. Nothing on that list is "teaching the model facts." Fine-tuning shapes how a model responds. It is a poor way to make it know something, because facts learned this way cannot be updated without retraining, cannot be cited, and blur into what the model already believed.

So "fine-tune it on our documentation" is usually the wrong instrument. When your documentation changes you retrain; with retrieval you re-index. When a user asks where an answer came from, retrieval can point at a document and a fine-tuned model cannot. RAG against fine-tuning covers the comparison in full.

The case flips when the problem is behavior. A model that will not stick to your output shape, or reply in your house style, or classify into your categories, is a behavior problem, and that is what fine-tuning is for.

Four methods behind one word

"Fine-tuning" hides several procedures with different data requirements.

The distinction matters before you collect data, because these want different data. Supervised training needs correct outputs. Preference optimization needs pairs, a better and a worse response to the same input, which is a different labeling job.

The order to work in: evaluate, prompt, then fine-tune

OpenAI's optimization workflow is: build evaluations, prompt the model with context and instructions, then consider fine-tuning for some use cases. The guide adds that "the prompt engineering process may be all you need in order to get great results for your use case."

Evaluations come first. Not second, and not at the end.

The reason is that fine-tuning without a measurement is unfalsifiable. You will have a model that behaves differently and no way to say whether it behaves better. Every later question, whether to keep it, whether to retrain after a base model update, whether the data helped, needs a number you can only get from a set you built beforehand. LLM evaluations covers building one.

So the order is: measure, prompt, retrieve, and only then fine-tune, with evidence at each step that the cheaper option was not enough.

What fine-tuning costs beyond the training run

The training run is the cheap part, and the part people budget for.

The expensive part is the dataset. Fine-tuning needs examples of the behavior you want, which means somebody writes or curates them, and quality matters more than volume: a few hundred consistent examples beat thousands of inconsistent ones, because the model learns the inconsistency too.

No source gives a minimum size, and any number quoted without your task would be invented. What the guidance does say is to evaluate on test data representative of real inputs, compare the prompted and fine-tuned model, and iterate.

The second cost is ongoing. A fine-tuned model is a fork. When the base model improves, you do not get the improvement until you retrain, and the evaluation has to be rerun to confirm the new version still behaves. That maintenance outlives the project that justified it, and it belongs in the decision rather than being discovered later.

Further reading

• OpenAI, Model optimization: the workflow ordering, the documented benefits, and the four methods.
• LLM evaluations: the measurement to build first.
• Self-hosted models: where modifying weights becomes practical.