What is a Model Parameters?

A model's parameter count (7B, 70B, 405B, etc.) is the number of these learned values, and it's a rough proxy for capacity: more parameters generally let a model represent more complex patterns, at the cost of more memory and compute to run. Parameters are distinct from hyperparameters, which are settings chosen before training (learning rate, batch size, number of layers) rather than learned from data. During inference, parameters are fixed, the model doesn't update them while answering a query. Fine-tuning and continued pretraining are the processes that change parameter values after the initial training run, whether through full fine-tuning (updating all parameters) or parameter-efficient methods like LoRA that only adjust a small added subset.