Unsupervised NLP Classification with Clustering
summary
Pipeline processes budget PDFs with NLTK tokenization, stemming and stopword filtering, builds minhash approximation indexes, computes Jaccard edges above a threshold, and clusters a NetworkX graph with greedy agglomerative clustering implementation. From ~26,000 filtered sentences it finds about ten multi-source topic clusters (sizes 5–100) and surfaces concrete topics like fund balance accounting and intergovernmental revenue sources. Code and data run end-to-end and produce actionable clusters despite noisy PDF text and basic sentence parsing.
AI-written summary · about
chapters (14)
- 0:00Introduction to OpenGov
- 1:42Challenges in supervised learning
- 4:51Clustering vs utility optimization
- 6:17Hierarchical clustering explained
- 9:05Live coding setup
- 11:11Processing government PDFs
- 13:39Text preprocessing with NLTK
- 25:34Sentence extraction and tokenization
- 28:05MinHash for similarity
- 29:27Greedy agglomerative clustering
- 31:11Visualizing clusters
- 33:17Interpreting results
- 35:49Improving the approach
- 36:11Q&A
related talks
26mUnsupervised learning for news summarisation
41mAutomatic Text Categorization
45mTransformer based clustering: Identifying product clusters f...
37mBuilding a Scientific Taxonomy at Scale
33mImpact of NLP in AI-powered Bank
01h 09mAutomatic Tagging of Short Texts with scikit-learn and NLTK
25mThe Art of Clustering: The Good, The Bad and The Beautiful
01h 38mNavigating the World of NLP and Text Analytics