What is a Neural Network?

Each node in a layer receives weighted inputs from the previous layer, applies an activation function that introduces non-linearity, and passes its output forward. Stacking many layers, a deep network, lets the model learn increasingly abstract representations, early layers in an image model might detect edges, later layers detect shapes, and the final layers detect entire objects. Training uses backpropagation: the network makes a prediction, an error is measured against the true answer, and that error is propagated backward through the layers to adjust weights via gradient descent, repeated over millions of examples. Modern large language models are a specific neural network architecture called the transformer, which uses a mechanism called attention to let every token in a sequence weigh the relevance of every other token, rather than processing text strictly left-to-right. This architecture is what made models capable of handling long-range context and scaling to hundreds of billions of parameters practical to train.