What is a Model Alignment?

A language model's raw pretraining objective, predicting the next token, doesn't by itself guarantee the model will be helpful, honest, or safe; it just gets good at continuing text in ways that resemble its training data. Alignment work closes that gap through additional training stages: supervised fine-tuning on curated examples of desired behavior, and reinforcement learning from human or AI feedback, which rewards outputs that match human judgments of quality and safety and penalizes ones that don't. Alignment isn't a single fixed target — it spans multiple, sometimes competing goals: being genuinely helpful and following instructions, refusing genuinely harmful requests, being calibrated about its own uncertainty, and avoiding sycophancy, meaning telling users what they want to hear rather than what's true. Getting the balance wrong in either direction causes real problems: an over-aligned-to-refuse model becomes uselessly cautious, while an under-aligned one becomes unreliable or unsafe in ways users can't predict. Alignment also extends beyond training into deployment-time techniques, such as system prompts, tool-use guardrails, and output filtering, since no amount of training alone perfectly constrains behavior across every possible input a model will encounter in production.