What is Few-Shot Learning?

Traditional machine learning needed thousands of labeled examples and a training run to learn a new task. Large language models changed that: because they've already learned broad patterns of language and reasoning during pretraining, showing them just two or three examples of the input-output format you want ("here's a question and a well-formatted answer, now do the same for this new question") is often enough for them to generalize to the pattern, with no weight updates involved. This is distinct from zero-shot learning, where no examples are given at all, and from fine-tuning, where the model's weights are actually updated using a larger dataset. Few-shot prompting is popular precisely because it's cheap and fast — no training pipeline, no GPU cluster, just careful example selection in the prompt — but it has limits. Performance is sensitive to which examples you choose and their order, it consumes context-window space that could otherwise hold task-relevant data, and for tasks with subtle or highly specific domain rules, fine-tuning or retrieval-augmented approaches often outperform it.