Neural networks · 4 of 5 · Python · With a year
Compose a multilayer perceptron
A multilayer perceptron, or MLP, is neurons arranged in layers. A hidden layer holds several neurons that all read the same inputs, each with its own weights and bias, and each gives one number. An output neuron then takes a weighted sum of those numbers. One layer feeding the next is all there is to it. With enough hidden neurons, this shape can fit almost any curve.
About this lesson
Words to know
- hidden layer
- A layer of neurons between the input and the output. Its values are not seen from outside.
- forward pass
- Running input through the model, layer by layer, to get its output.
- ReLU
- The function
max(0, x). It keeps positive numbers and turns everything else into 0.
Why it matters
Every transformer block in a GPT ends with a small MLP like this one. The digit classifier in chapter 4 is one layer of it.
With a year
How it works, a worked example, the exercise with its tests, three hints and the explained solution are in the app, with a year of Algo150 for $19. The first chapter, Math foundations, is free for everyone: start with Follow a gradient.