Detecting novel anomalies in Twitter
summary
Anomaly detection flags finance-relevant Twitter spikes by modeling tweet counts per time window with Poisson regression with time‑of‑day day‑of‑week and recent‑count predictors then scoring low tail probabilities against an alpha threshold, switching to negative‑binomial or zero‑inflated when overdispersion or many zeros appear. Novelty filtering uses Locality‑Sensitive Hashing plus a recent‑tweet inverted index to emit a single representative, novel tweet. Code is in Python using scikit‑learn and statsmodels and the Volkswagen emissions spike is shown as a case.
AI-written summary · about
chapters (9)
- 0:00Detecting novel anomalies in Twitter
- 4:00Why anomaly detection matters
- 7:40Processing tweets for content
- 10:36Anomalous tweet volumes
- 16:51Modeling tweet volume with Poisson
- 21:44Novelty detection in anomalies
- 27:24Two-step novelty approach
- 33:41Limitations and future work
- 39:51Spam and credibility in tweets
related talks
18mCross-Industry Anomaly Detection Solutions with R and Shiny- 18mCross-Industry Anomaly Detection Solutions with R and Shiny
28mUnsupervised Real-Time Stream-Based Novelty Detection Technique
43mDetecting Anomalies in Streaming Data
45mAnomaly Detection in Time Series
52mDevil in the Haystack
30mMassive Scale Anomaly Detection Framework
01h 16mAnomaly Detection