Build a digit classifier head
A classifier for handwritten digits takes an image as a list of numbers and gives ten scores, one for each digit from 0 to 9. The digit with the highest score is its answer. The part that turns numbers into scores is one linear layer: the model from chapter 1, but with one column of weights per class instead of a single list. This lesson builds that last layer and reads off its answer.
About this lesson
Words to know
- logit
- A raw score a model gives to one choice, before the scores are turned into probabilities.
- argmax
- The position of the largest value in a list.
- feature
- One of the numbers that describe an example, such as one pixel of an image or one word count.
- dot product
- Multiply two lists of numbers position by position and add up the results. It gives one number.
Why it matters
A GPT ends the same way: a linear layer that gives one score per token in the vocabulary. Picking the largest is the greedy decoding you write in chapter 7.
With a year
How it works, a worked example, the exercise with its tests, three hints and the explained solution are in the app, with a year of Algo150 for $19. The first chapter, Math foundations, is free for everyone: start with Follow a gradient.