Vector search is fast because it compares two embeddings that were computed separately. The query never actually meets the document. That independence is what allows a million documents to be searched in milliseconds, and it is also why the top ten results are often in the wrong order.
A reranker fixes the order. It is usually a cross-encoder: a smaller transformer that reads the query and one candidate document together and outputs a single relevance score. Because both texts share the same attention pass, the model can see whether the document answers this particular question rather than whether it sits in a similar region of vector space.
The standard pattern retrieves fifty to a hundred candidates cheaply, reranks them, and passes the top five into the prompt. Cost stays bounded because the expensive model only ever sees a short list.
In retrieval-augmented systems this single stage is often the largest quality gain available. Teams that report weak RAG results are frequently retrieving the right document and burying it at position nine.