Fit a context window
A model reads everything you send it in one piece, called the context window, and that window has a fixed size measured in tokens. The instructions, the conversation, the retrieved chunks and the answer the model is about to write all have to fit. So room for the instructions and the answer is set aside first, and the chunks are fitted into what is left. A chunk that does not fit is skipped, and smaller ones after it may still fit.
About this lesson
Words to know
- context window
- Everything a model can read and write in one request, measured in tokens. It has a fixed maximum size.
- token
- The unit a model reads and writes: a word, a piece of a word, or a punctuation mark. Limits and prices are counted in tokens.
- token budget
- The number of tokens you allow one part of a prompt to use.
- chunk
- A piece of a document, small enough to search and to fit in a prompt.
Why it matters
More context is not better by default: text that does not help costs money and distracts the model. A budget forces a choice, and the order of the chunks decides who wins.
With a year
How it works, a worked example, the exercise with its tests, three hints and the explained solution are in the app, with a year of Algo150 for $19. The first chapter, Model foundations, is free for everyone: start with Vector dot product.