Dimension Reduction and Extracting Topics - A Gentle Introduction
summary
Explains singular value decomposition (SVD) and uses a term‑document matrix with tf‑idf weighting to extract topics from real datasets, including the Chilcot Iraq Report (2.7M words) and messy Hillsborough OCR. Reimplements algorithms from scratch, publishes notebooks on GitHub, and finds a small number of dominant singular values (roughly 4–10) that produce coherent topics — e.g., treaties, veterans/medical issues, telegrams — working well on clean text but degrading with noisy OCR.
AI-written summary · about
chapters (15)
- 0:00Topic extraction intro
- 1:25Speaker background and meetup groups
- 2:10Text mining challenges
- 4:36Word similarity in text
- 7:25Defining topics
- 9:31Term-document matrix
- 12:46Dimensionality reduction for themes
- 15:13Singular Value Decomposition explained intuitively
- 21:58SVD for topics
- 25:51SVD example
- 27:14TF-IDF processing
- 30:24Data cleaning importance
- 31:29Topics from Iraq report
- 33:20Applications and limits
- 34:16Q&A
related talks
49mTopic Modeling: A Provable Spectral Method
56mTensor Decompositions for Learning Latent Variable Models I
31mA practical guide to Singular Value Decomposition in Python
34mDimensionally reducing data squeezing out the good stuff
40mA Practical Guide to Dimensionality Reduction Techniques
13mThinking of Topic Modeling as Search
45mWords in Space: A Visual Exploration of Distance, Documents, and Distributions...
37mA Bluffer's Guide to Dimension Reduction