When Rotten Tomatoes Isn’t Enough: Analyzing Twitter Movie Reviews
summary
Pipeline uses Apache Cassandra (DataStax Enterprise Analytics) as the store, Apache Spark to tokenize and remove stop words, Python + Jupyter notebooks to glue things, and tweets from the Twitter API (demo uses pre‑pulled CSVs) inserted via Docker containers. Spark frames convert to pandas then pattern sentiment scores classify tweets and an average positive vs negative score yields a recommendation. A surprising failure mode: the word “impossible” skews scores; adding custom stop words corrects it. The demo runs end‑to‑end and requires tuning for scale and accuracy.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 1:17Problem overview and demo goals
- 3:03Apache Cassandra
- 11:06Why use Cassandra
- 11:47Apache Spark
- 13:57Sentiment analysis
- 16:26Demo setup
- 17:52Jupyter notebook walkthrough
- 31:39Sentiment results
- 53:08Docker and Jupyter setup
- 1:11:16Twitter API demo
- 1:13:20Q&A
related talks
34mActionable Insights with Real-time Streaming Analytics of Customer Reviews
28mAn Architecture to Tweet Them All
43mDetecting fake reviews at scale using Apache Spark and John Snow Labs
01h 26mData Driven Action: a primer on Data Science
01h 47mRadanalytics.io streaming applications workshop
01h 25mApache Spark for Big Data Processing
26mReview Analysis: an Approach to Leveraging User-Generated...
46mSparking Success: Unveiling the Journey of Apache Spark Application Development