What is Transfer Learning?

Instead of initializing a model's parameters randomly, transfer learning starts from a model that has already learned general-purpose representations. For language models this means broad grammar, world knowledge, and reasoning patterns learned during large-scale pretraining. A smaller, task-specific dataset is then used to fine-tune that base model, nudging its parameters toward the new task, such as a particular writing style or a company's internal terminology, without erasing what it already knows. This approach is what makes most practical AI development affordable: very few organizations can pretrain a foundation model from scratch, but many can fine-tune or adapt an existing one for a specialized purpose with a modest dataset. The tradeoff is that fine-tuning too aggressively on narrow data can cause catastrophic forgetting, where the model loses some of its general capability in exchange for improved performance on the narrow task.