Underfitted

Share this post

Neural Networks Hate Text

underfitted.svpino.com

Neural Networks Hate Text

An introduction to word embeddings, one of the fundamental ideas behind generative AI models.

Mar 3, 2023
∙ Paid
14
Share this post

Neural Networks Hate Text

underfitted.svpino.com
2
Share

The Internet is mainly text.

For centuries, we’ve captured most of our knowledge using words, but there’s one problem:

Neural networks hate text.

Judging by how good language models are today, this might not be obvious, but turning words into numbers is more complex than you think.

The most straightforward approach is to use consecutive values to represent …

Keep reading with a 7-day free trial

Subscribe to

Underfitted
to keep reading this post and get 7 days of free access to the full post archives.

Already a paid subscriber? Sign in
Previous
Next
© 2023 Santiago L. Valdarrama
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing