Overlapping document chunks
A model can only read a limited amount of text at once, and a search works better on small pieces than on whole documents. So before a document is stored for search, it is cut into pieces called chunks. A clean cut can split a sentence in two, leaving half of a fact in each piece. To soften that, each chunk repeats the last few words of the one before it. That repeated part is the overlap.
About this lesson
Words to know
- chunk
- A piece of a document, small enough to search and to fit in a prompt.
- overlap
- The words at the end of one chunk that are repeated at the start of the next.
- RAG
- Retrieval-augmented generation: find the documents that matter for a question, and give them to the model together with the question.
Why it matters
Chunks are what a retrieval system stores, ranks and hands to the model. Chunk size and overlap change which facts can be found, so the roadmap asks you to measure them, not guess.
With a year
How it works, a worked example, the exercise with its tests, three hints and the explained solution are in the app, with a year of Algo150 for $19. The first chapter, Model foundations, is free for everyone: start with Vector dot product.