Late chunking encodes a whole document before splitting its token representations into searchable chunks, so each chunk keeps information from the surrounding document. It improves retrieval for passages that rely on a definition, subject, or earlier section that would disappear if the chunk were embedded alone.
Jina AI introduced the term for an embedding approach that delays splitting until after the model has processed the complete document. That makes the method a response to lost context, not a universal replacement for document chunking.
Why can late chunking improve retrieval?
A normal embedding for a small passage may not know who “they” means or which product a price refers to. Late chunking lets the encoder use the surrounding document before it creates that passage vector. The stored chunk retains document-level meaning without storing the whole document as one result.
When is late chunking worth the cost?
It needs an encoder that can process enough of the source document at once, so long documents may still require a different approach. Use it when retrieval tests show that isolated chunks lose important references.Contextual retrieval solves a similar problem by adding an explanatory prefix to each chunk. Both can improve RAG, but neither replaces evaluation against the questions users actually ask.
Frequently asked questions
How is late chunking different from normal chunking?
Normal chunking splits text before embedding it. Late chunking encodes the full document first, then pools the contextual token representations for each chunk.
Does late chunking remove the need for document context at retrieval time?
No. It improves each chunk's embedding, but the system still needs sensible chunk boundaries, source citations, and enough surrounding text for the model to answer safely.
No advertising or tracking cookies, and our visitor counts are anonymous. The Cal.com booking widget loads only if you allow it. Privacy Policy.
The page itself, anything our host sets to serve and secure it, and the anonymous visitor count. Always on, and none of it stores anything on your device.
The Cal.com booking widget. Left off, a booking link opens the booking page instead of a popup, so you can still book a call.