Languages as designed latent spaces
This post is an edited version of a Mastodon thread I wrote recently. I have been thinking recently about how language is a high entropy latent space and what that means. Latent spaces are sort of magic: they don’t just reduce the dimensionality of a higher dimensional space, they also constrain it such that arbitrary movement in the latent space maps to some meaningful position in the higher dimensional space.
- ▪This post is an edited version of a Mastodon thread I wrote recently.
- ▪I have been thinking recently about how language is a high entropy latent space and what that means.
- ▪Latent spaces are sort of magic: they don’t just reduce the dimensionality of a higher dimensional space, they also constrain it such that arbitrary movement in the latent space maps to some meaningful position in the higher dimensional spa
2 outlets in our directory ran this story, first to last over 13 hours. All of the coverage we found sits in one bucket: centre. That one-sidedness is itself worth noticing.
- ▪ Languages as designed latent spaces — Lobsters
Lobsters files mainly under programming. We currently carry 170 of its stories.
Opening excerpt (first ~120 words) tap to expand
This post is an edited version of a Mastodon thread I wrote recently. I have been thinking recently about how language is a high entropy latent space and what that means. Latent spaces are sort of magic: they don’t just reduce the dimensionality of a higher dimensional space, they also constrain it such that arbitrary movement in the latent space maps to some meaningful position in the higher dimensional space. Think about how almost any textual prompt you give to an image generator results in a picture with recognisable shape and form, yet the vast majority of the possible images are basically just random noise. Language is, obviously, like this.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at Jsbarretto.
Discussion
0 commentsMore from Jsbarretto
