What is a Large Language Model?

Modern LLMs are built on the transformer architecture, which uses self-attention to weigh the relevance of every prior token when generating the next one, allowing the model to track long-range context across a conversation or document. Training happens in stages: pretraining on trillions of tokens of general text teaches broad language and world knowledge, and subsequent fine-tuning (including reinforcement learning from human feedback) shapes the model to follow instructions, refuse harmful requests, and produce helpful, well-formatted answers. LLMs are probabilistic next-token predictors, not databases — they can produce fluent, confident text that is factually wrong (hallucination), and their knowledge is frozen at the training cutoff unless it's supplemented with retrieval or tools. This is why production systems pair an LLM with grounding mechanisms like a knowledge base, tool calls, or structured retrieval rather than trusting the model's raw output for anything that must be verifiably accurate.