Data driven literary analysis
summary
Works from 28 Shakespeare plays with ~11,000-word vocabulary, uses tf-idf to score terms, reduces dimensionality with LDA topic modelling (gensim) to represent each play as a mixture of topics, then applies k-means clustering (sklearn). Topic lists show clear themes labeled death, love and hero; clustering does not recover comedy vs tragedy (Romeo and Juliet lands with comedies) and instead reveals a temporal/maturity split between early and later plays. Slides and code are provided.
AI-written summary · about
chapters (11)
- 0:00NLP project on Shakespeare
- 3:11Supervised vs unsupervised NLP
- 5:41TF-IDF feature extraction limits
- 8:16Topic modeling with LDA
- 11:55Two-stage analysis: topics + clustering
- 14:23Text preprocessing for analysis
- 16:27Interpreting LDA results on Shakespeare
- 19:21Clustering insights
- 21:10Temporal stylistic patterns
- 23:17Conclusions from project
- 24:58Q&A
related talks
41mUsing Clojure+Spark to Find All the Topics on the Interwebs
25mMachine Learning for source code analysis
35mHow does text become data?
01h 01mWorkshop: Intro to topic modeling using Python
45mDecoding Topics: A Comparative Analysis of Python’s Leading Topic Modeling Libraries Using Climate C
45mEvaluating Topic Models
28mA new wave of Network Science - from fantasy to applications
46mPractical Machine Learning and Rails