Word Embedding For Fun and Profit

on YouTube ↗01h 03m

" summary

Trains gensim word2vec on a ~170k‑word movie‑plots corpus from the movie-plots-by-genre repo, compares baselines: majority label 40%, tf‑idf bag‑of‑words 46%, doc2vec 52%, and simple averaged word vectors wins (~58%) out of the box. Notes: training is fast (~25s on laptop for this small set), prediction for doc2vec requires descent at inference and is unstable, models need ~500k+ words to stabilize, removing stopwords and PCA help, and real gains require careful hyperparameter tuning.

AI-written summary · about

" chapters (14)

1. 0:00Word embeddings and GenSim
2. 2:34Theory behind embeddings
3. 10:33How word2vec works
4. 22:05Movie genre demo setup
5. 24:16Baseline genre models
6. 29:14Word vectors for classification
7. 33:09Doc2Vec for genre prediction
8. 43:47Model evaluation limits
9. 48:14Debugging word embeddings
10. 53:51Embeddings vs other NLP
11. 55:37Bias and fairness in embeddings
12. 57:11Analogies and multilingual embeddings
13. 1:00:18Randomness in production
14. 1:02:47Closing and Q&A

" related talks

• 34mWord Embeddings for fun and profit in GensimLev Konstantinovskiy · PyData London 2016
• 01h 01mWord Embeddings - An Alternative & Efficient Approach to Search for DocsAnanth Gundabattula · Developer Summit 2020
• 46mBeyond Bag of Words A Practitioner’s Guide to Advanced NLPAriel M’ndange-Pfupfu, Marc Anderson · PyData DC 2016
• 59mContextual search with vector search: exploring your options with open source toolsOlena Kutsenko · NDC Oslo 2024
• 39mBeyond word2vec: GloVe, fastText, StarSpaceKonstantinos Perifanos · PyData London 2018
• 40mText similiarity with the next generation of word embeddings in GensimLev Konstantinovskiy · PyData Berlin 2017
• 42mDocument Vectors in the WildJames Dreiss · Strata Data Conference 2018
• 35mHow does text become data?Catherine Havasi, Rob Speer · Strange Loop 2013