Build and Deploy an End-to-end Streaming NLP Insight System
summary
Builds a working end to end pipeline that ingests live Reddit comments, serializes them to JSON, pushes them through uses Redis as message queue, runs spaCy for named entity extraction and a FastAI/PyTorch sentiment classifier (300 MB download), and publishes annotated sentences to a mobile bot. Code is containerized with Conda and Docker Compose, packaged as pip-installable modules, and serves results to Telegram mobile; the demo runs live and the orchestrated stack starts with one command.
AI-written summary · about
chapters (12)
- 0:00Welcome
- 5:28Reddit data consumer
- 18:23Redis message queue
- 22:47Text preprocessing
- 31:20NLP with Spacy
- 36:50Named entity recognition
- 40:15Sentiment analysis
- 47:59Code modules
- 57:01Telegram bot
- 1:06:10Docker Compose
- 1:16:49Live demo
- 1:20:30Next steps
related talks
41mAdding Generative AI to Real Time Streaming Pipelines
43mSentiment Analysis in Action: Building Your Real-time Pipeline
26mBuilding A Python Service Using Open Source Components That Connect, Transform & Deliver Data In RT
21mFrom 0 to a crowd sourced neural network in a jiffy
30mDeploy end to end ML pipeline using Apache spark ...
45mArchitecture of an NLP Deployment
01h 47mRadanalytics.io streaming applications workshop
01h 27mDeprecating the state machine: building conversational AI with the Rasa stack