The Art of Clustering: The Good, The Bad and The Beautiful
summary
Clustering workflow uses sentence-transformer embeddings, UMAP for dimensionality reduction, HDBSCAN for clustering and datamapplot/ Bokeh for interactive exploration, with CLIP used for poster embeddings; a 5,000-film run yields 47 clusters and 48% noise, showing huge unlabeled mass. LLMs augment labels (e.g., “coming of age drama”) but produce errors like mixing pre‑WWI films into World War II clusters; guidance covers parameter effects, hierarchical subclusters, and when clusters should not be used to auto‑label training data.
AI-written summary · about
chapters (20)
- 0:00Clustering challenges intro
- 1:05Clustering answers unasked
- 2:30Clustering pipeline explained
- 3:56Clustering creates structure
- 5:21Choosing embedding method
- 7:06UMAP dimensionality reduction
- 8:13HDBSCAN results and noise
- 9:58LLM representation pitfalls
- 12:09Exploration vs classification
- 13:34LLMs for clustering augmentation
- 14:58Visualizing hierarchical clusters
- 16:07Clusters to classifiers
- 17:10Multimodal text-image clustering
- 17:31Clustering truth takeaways
- 18:35Parameters and consequences
- 19:19Pipeline not a button
- 20:23Handling clustering noise
- 21:25Clustering visualization tools
- 22:54Choosing clustering algorithm
- 24:18Q&A
related talks
38mData Mapping for Data Exploration
28mUnderstanding UMAP Internal Workings of a State of the Art Clustering Algorithm
35mClustering: A Guide for the Perplexed
40mTopological Techniques for Unsupervised Learning
27mUnveiling Clustering in BERTopic Topic Modeling
27mClustering of high-content screen images to discover off-target phenotypes
45mTransformer based clustering: Identifying product clusters f...
32mBigData Clustering Illustrated by OK Gifts