What is a Small Language Model?

Where large language models chase general-purpose capability across nearly any task, SLMs trade some of that breadth for speed, lower memory footprint, and the ability to run on-device or on modest servers without a GPU cluster. Techniques like distillation (training a small model to mimic a large one's outputs), pruning, and quantization let SLMs retain much of the accuracy on focused tasks — classification, extraction, routing, simple summarization — while costing a fraction of the inference price. The practical appeal is latency and cost control: an SLM can screen or pre-process requests in milliseconds, reserving the expensive large model for the subset of work that actually needs deep reasoning. This tiered approach is increasingly common in production AI systems, where a cheap model handles triage and a frontier model handles the hard cases it escalates.