Sequence Generation

Sequence Generation


Sequence generation is a model producing its output piece by piece rather than all at once. In language models those pieces are tokens. The model picks the first token, appends it to the input, picks the second, and keeps going. Every choice is made while looking at everything written so far.

The pick is probabilistic rather than fixed. At each step the model produces a probability distribution over the whole vocabulary, and the decoding rules decide which token to draw from it. Lowering temperature pushes the most likely token forward, raising it widens the variety. Methods like beam search track several candidate sequences in parallel instead of committing to one path.

This is not limited to text. Machine translation, summarisation, code completion, speech synthesis and music generation all work the same way. What they share is an output with an order, where each step depends on the ones before it.

A worked example: while drafting a product description the model picks the token "These", and the odds of "shoes" coming afterwards rise because the sentence now has a subject. Running the same prompt again can produce a different sentence, since the choice is resampled at every step.

Because the process is stepwise, mistakes compound. A sequence that turns the wrong way early tends to build on that turn rather than correct it later.

From generative AI strategy to custom agent development and retrieval architectures, we help you scale AI responsibly.
Discuss your AI project