NLP · 2 of 4 · Python · With a year
Represent text with word counts
The oldest way to turn text into numbers is to count words. Pick a list of words you care about, the vocabulary. For a piece of text, count how many times each of those words appears. The text becomes a list of counts, always the same length, which a simple model can use. Word order is thrown away, which is why this is called a bag of words.
About this lesson
Words to know
- vocabulary
- The full list of tokens a model knows. A token's place in the list gives its ID.
- bag of words
- A text described only by how often each word appears. Word order is ignored.
- feature
- One of the numbers that describe an example, such as one pixel of an image or one word count.
- baseline
- A simple model that sets the score a more complex model has to beat.
Why it matters
Counts like these are a cheap baseline to beat. The sentiment lesson next uses the same split, and its weakness is why models moved on to embeddings and attention.
With a year
How it works, a worked example, the exercise with its tests, three hints and the explained solution are in the app, with a year of Algo150 for $19. The first chapter, Math foundations, is free for everyone: start with Follow a gradient.