What is Fine-Tuning?

A foundation model is trained on broad, general-purpose data and learns general capabilities, but it may not be optimally calibrated for a narrow task — a specific writing style, a proprietary classification scheme, or domain jargon it saw rarely during pretraining. Fine-tuning addresses that by running additional training passes on a curated dataset of examples specific to the target task, updating the model's weights (fully, or more commonly today via parameter-efficient methods like LoRA that adjust only a small added set of parameters) so its behavior shifts toward the desired pattern. Fine-tuning sits above few-shot prompting and below training a model from scratch on the effort-versus-control spectrum: it requires real training infrastructure and a labeled dataset, but produces a model that reliably exhibits the target behavior without needing examples repeated in every prompt, and it can bake in patterns too subtle or too numerous to fit in a context window. The risks are real too — a poorly curated fine-tuning set can degrade the model's general capabilities (catastrophic forgetting) or bake in biases present in the narrow dataset, so evaluation against held-out data matters as much as the training itself.