Autoregressive generation is serial by nature. Each token waits for the one before it, and every step costs a full forward pass through the large model. Speculative decoding breaks that serial chain without changing what the model would have said.
A small draft model, often ten to twenty times cheaper, proposes the next four or five tokens. The large model then scores that whole span in one forward pass, because verifying a known sequence is parallel work in a way that generating it is not. Where the large model agrees with the draft, all those tokens are accepted at once. At the first disagreement the draft is truncated and the large model supplies the correct token itself.
The acceptance rate drives the speedup. On predictable text such as boilerplate code or formatted output, drafts survive often and latency can fall by half. On dense reasoning, acceptance drops and the gain shrinks.
The important property is that the sampling distribution is preserved exactly. This is a latency optimisation, and it costs nothing in quality.