I audited Stanford's CS336 and built an LLM from scratch for $353
If you wish to make an apple pie LLM from scratch, you must first invent the universe systems considerations. — Carl Sagan, probably In December, I finished working through Stanford NLP’s CS336: Language Models From Scratch. This post is a short review of the class, and a bunch of practical tips/considerations if you’re considering auditing it as well. The basic premise of the course is that: Researchers/Engineers are becoming disconnected from the underlying details of LLMs.
- ▪If you wish to make an apple pie LLM from scratch, you must first invent the universe systems considerations. — Carl Sagan, probably In December, I finished working through Stanford NLP’s CS336: Language Models From Scratch.
- ▪This post is a short review of the class, and a bunch of practical tips/considerations if you’re considering auditing it as well.
- ▪The basic premise of the course is that: Researchers/Engineers are becoming disconnected from the underlying details of LLMs.
Opening excerpt (first ~120 words) tap to expand
If you wish to make an apple pie LLM from scratch, you must first invent the universe systems considerations. — Carl Sagan, probably In December, I finished working through Stanford NLP’s CS336: Language Models From Scratch. This post is a short review of the class, and a bunch of practical tips/considerations if you’re considering auditing it as well. The basic premise of the course is that: Researchers/Engineers are becoming disconnected from the underlying details of LLMs. Moving up these levels of abstraction can allow you to move faster. BUT: There are elements of deeper understanding, research taste, and implementation prowess that are increasingly easy to never learn.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at Andy Timm.
Discussion
0 commentsMore from Andy Timm
