Reranking is a second retrieval stage that takes a small set of initially retrieved documents and scores them again with a more accurate model. It improves which passages reach an LLM by spending extra computation only on the leading candidates rather than searching the entire corpus with the expensive model.
Microsoft’s semantic ranker documentation shows this two-stage pattern: retrieve candidates first, then rescore the top set for semantic relevance.
How does reranking fit into a RAG pipeline?
Semantic search or hybrid search produces the candidate set. The reranker compares each candidate more closely with the query and passes only the best passages onward. The second pass spends accuracy where it can still change the answer.
When is reranking worth the latency?
Add it when initial retrieval finds relevant material but ranks it too low. It cannot recover a passage that the first stage never retrieved. Use RAG evaluation to compare relevance and answer quality before and after the change. A larger candidate set may improve recall, but it also increases cost and can crowd the context window with near duplicates.
Frequently asked questions
What is a reranker in RAG?
A reranker is a model or scoring service that compares the query with each candidate passage and produces a new relevance order before context is sent to the generator.
Does reranking make retrieval slower?
Yes. It adds a second scoring pass, so teams trade added latency and cost for better ordering of the small candidate set.
No advertising or tracking cookies, and our visitor counts are anonymous. The Cal.com booking widget loads only if you allow it. Privacy Policy.
The page itself, anything our host sets to serve and secure it, and the anonymous visitor count. Always on, and none of it stores anything on your device.
The Cal.com booking widget. Left off, a booking link opens the booking page instead of a popup, so you can still book a call.