Supervised Fine-Tuning

Supervised Fine-Tuning


Supervised fine-tuning adapts a pre-trained model to a specific task or a specific way of behaving, using labelled examples. The model already knows the language. What it learns here is what the desired output should look like.

The data is made of input and expected-output pairs. For chat assistants those pairs are example dialogues written by people: what a user asks and what a good answer looks like. In task-focused adaptation the pairs come straight from the work, such as a contract and the fields that should be extracted from it. Training can update all the model's weights, or methods like LoRA can train only a small added layer.

Inside an RLHF pipeline this is the opening stage. Supervised fine-tuning brings the model toward the desired shape first, and a reward model trained on preference data takes over afterwards.

A concrete case: an insurer builds an assistant that answers policy questions. The general model is accurate but writes long, generic replies. Fine-tuning on 2,000 question and answer pairs written by the company's own support team makes the responses shorter and moves them onto the company's terminology.

The risk is narrowing. Fine-tuning on a small, uniform set can make the model worse on questions outside that set, so general capability gets checked against a separate evaluation.

From generative AI strategy to custom agent development and retrieval architectures, we help you scale AI responsibly.
Discuss your AI project