Neural Networks Hate Text
An introduction to word embeddings, one of the fundamental ideas behind generative AI models.
The Internet is mainly text.
For centuries, we’ve captured most of our knowledge using words, but there’s one problem:
Neural networks hate text.
Judging by how good language models are today, this might not be obvious, but turning words into numbers is more complex than you think.
The most straightforward approach is to use consecutive values to represent …
Keep reading with a 7-day free trial
Subscribe to
Underfitted
to keep reading this post and get 7 days of free access to the full post archives.