The transformer decoder is the half of the architecture that produces output. Its job is to predict the next token in a sequence from everything available so far. Models in the GPT family are built from this half alone, so how the decoder behaves is how generative models behave.
Masking is what separates a decoder from an encoder. Each token may only attend to the tokens before it, and anything later is hidden. Without that restriction the model would copy the answer during training and never learn to generate. In translation models that use both halves, a second attention layer sits in the decoder and looks at the encoded representation of the source text.
Generation proceeds a step at a time. The decoder emits a token, appends it to the input, and the loop starts again. To avoid redoing the same arithmetic at every step, the key and value representations from earlier steps are kept in a cache, and on long responses most of the speed comes from there.
A worked example: in a code completion tool a developer types for i in ra and the decoder, reading the file so far plus the current line, produces nge( with high probability. Each additional character narrows the next prediction, so suggestions sharpen as the line grows.
Because of masking a decoder cannot go back and revise its own output. A poor decision made early keeps shaping the rest of the sequence.