Measure retrieval recall
After changing a search, how do you know it got better? Looking at a few answers is not enough: a convincing demo can hide a worse system. You need a number. For each test question you write down which documents hold the answer. Then you check how many of them the search returned among its first k results. That share is called recall at k: 1 means every needed document was found, and 0 means none was. Over a whole set of questions, you report the average.
About this lesson
Words to know
- evaluation set
- A fixed list of test questions with their correct answers or documents, used to measure a system the same way every time.
- relevant document
- A document that holds the answer to a test question.
- recall at k
- The share of the relevant documents that appear in the first k results.
Why it matters
The first portfolio project reports this number before and after each change, and once at the end on questions it kept aside. It measures the search only: an answer can still be wrong when recall is 1.
With a year
How it works, a worked example, the exercise with its tests, three hints and the explained solution are in the app, with a year of Algo150 for $19. The first chapter, Model foundations, is free for everyone: start with Vector dot product.