A transformer is the neural network architecture built for sequences, and it underpins today's language models. It arrived with the 2017 paper "Attention Is All You Need". What sets it apart from the recurrent networks before it is that it looks at all tokens at once instead of reading the text in order.
Self-attention sits at the centre. Each token computes how much weight to give every other token in the sequence and updates its representation accordingly. That is how a word's meaning gets captured even when it depends on another word far away in the sentence. Order is supplied separately through positional encoding, since attention on its own has no notion of sequence.
Running in parallel is what let training spread across GPUs, which opened the door to much larger datasets and much larger models. The architecture now carries images, audio, code and protein sequences as well as text.
A concrete case: in "Ali gave the book to Ayse, because she had already read it", the pronoun refers back across several words. A model reading strictly in sequence loses that link as the distance grows, while a transformer relates the two directly.
The cost lives in attention. Work grows with the square of sequence length, so long context stays expensive, and that is the technical reason behind context window limits.
