Un Enfoque Pythonico para Minería de Textos de Historias de Trabajo Arduo, Migración y Resiliencia
summary
Uses Python with Beautiful Soup, Sentence Transformers and BERTopic to mine the Bracero History Archive of ~3,000+ oral items; validates robots.txt, locates the highest item id via exponential then binary then linear search, and removes interviewer lines heuristically. Scrapes 3–3,308 ids, speeds extraction from over an hour to ~12 minutes with 10 workers, clusters into 20 topics, stores citations per interview, and exposes results in a live Shiny app with code on GitHub.
AI-written summary · about
chapters (6)
- 0:00Introduction to narrative analysis
- 4:31Study questions with Python
- 7:15Repository structure and scraping
- 14:25Text analysis with BERTopic and Sentence Transformers
- 23:27Conclusions and reflections
- 24:41Q&A
related talks
32mWeb Scraping Best Practises
01h 51mNatural Language Processing in Python
22mDistributed Web Scraping in Python
01h 27mTips, Tricks and Topics in Text Analysis
24mDistributed Web Scraping in Python
20mPython In The Job Market
31mBeginning your own data engineering projects
28mLifecycle of Web Text Mining: Scrape to Sense