What is an Attention Mechanism?

Before attention, sequence models like RNNs processed text step by step, compressing everything seen so far into a fixed-size hidden state — a bottleneck that made it hard to retain information from far earlier in a long input. Attention instead lets each position in the sequence directly compute a weighted combination of every other position, so a word late in a document can draw directly on a word from the very beginning without that information having to survive being compressed through every step in between. The transformer architecture — the basis of virtually every modern large language model — is built around "self-attention," where the model computes, for each token, how much it should attend to every other token, using learned query/key/value projections. This is what lets transformers process an entire input in parallel rather than sequentially, which is also why they scale so much better on modern GPU hardware than older recurrent architectures did. Attention is also the mechanism behind context length limits and their cost: naive self-attention cost grows quadratically with sequence length, which is exactly why longer-context models require specialized, more efficient attention implementations rather than just scaling the naive version up.