What is Unsupervised Learning?

Unlike supervised learning, where each training example comes with a known correct label the model is taught to predict, unsupervised methods are given only raw data and must discover structure on their own, whether by clustering similar items together, reducing high-dimensional data down to its most important patterns, or, in the case of large language models, learning to predict the next token in naturally occurring text, which requires no human-labeled examples at all since the correct answer for each position is simply the word that actually appears next. This matters practically because labeled data is expensive and slow to produce at scale, while raw text, images, and logs are abundant. The pretraining phase of essentially every modern large language model is a form of unsupervised, or self-supervised, learning over massive unlabeled text corpora, and it's what gives these models their broad language ability before any task-specific fine-tuning is applied.