What are embeddings?

If you've read the paper and have more questions than answers or are dying to read more about embeddings, you can read the bibliography. I've also picked out a curated shorter reading list to get you started. If you're looking for how to learn machine learning, you can check out my post on how I do it here. Also check out my ML notes.

Embeddings

• Word Embeddings - A Survey
• Dense Vector Embeddings
• Google's Short Embeddings Course
• Embeddings learned by matrix factorization

Recommender Systems

• Recommender Systems - A Primer - This covers a comprehensive survey of recommender systems from two of the most senior people in the field.
• Practical Recommender Systems - Great Walkthrough Book with Code
• Computing Taste: Algorithms and the Makers of Music Recommendation - Excellent book surveying music recommendations and the history of the field
• Survey Paper on Deep Learning in Recommendations
• The difference between search and recommendation
• System Design for recommendation and search

Vector Databases

• ANN Benchmarks - Website with benchmark algos on approximate nearest neighbors search
• Faiss similarity search

LLMs

• Understanding Large Language Models -- A Transformative Reading List - Sebastian's whole site is very worth reading, start with this survey of LLM posts and literature
• A Primer on Neural Network Models for Natural Language Processing - Good idea to read everything Yoav has written but this is a great start
• Figures Everyone Should Know
• Transformers from Scratch - This is the one I come back to every time.
• Illustrated Word2Vec - Jay's site is extremely good, this one is particularly good for Word2Vec
• Attention? Attention! - Deep dive into the attention mechanism.
• A History of NLP - Great summary of the field over the last 20 or so years.
• Dive into Deep Learning Course

Machine Learning Engineering

• Machine Learning Design Patterns - Fantastic book covering a variety of engineering considerations, including embeddings
• Systems Performance - From the guy who made flame charts, an amazing resource on engineering best practices
• Notes on NLP Projects - Criteria for starting a new one
• Operating Systems: Three Easy Pieces - A classic for diving deep into the stack
• The Hardware Lottery - Understanding why we develop neural nets the way we do, an extension of The Bitter Lesson
• Efficiently training on multiple GPUs