A brief history of fighting spam at Pinterest
summary
History of Pinterest spam defense traces a move from manual "shift-click" removals to a centralized spam service, an asynchronous pipeline called stingray async, and a synchronous rule engine that evaluates 1.3 billion events per day with ~100 live rules. The stack uses Kafka, Storm/SQS and Spark, a Python DSL for fast rule development, a rewind system to replay historical events and mass-undo actions. Results show reported spam reduced to noise and about 1% of users seeing spammy experiences, but rule development and backtesting remain priorities.
AI-written summary · about
chapters (20)
- 0:00Start
- 1:43Why spammers love Pinterest
- 3:02What spam on Pinterest looks like
- 3:57Our evolving approach
- 6:33Starting architecture (March '13)
- 6:49SpamService (June '13)
- 7:14Stingray Async (Sep '13)
- 8:24Stingray Sync (June '14)
- 8:46Scale
- 9:31Pin reports
- 10:34Strategic themes
- 13:45Timeline of an attack
- 16:44Rule development, pt. 2
- 18:46Improving post-attack cleanup
- 20:06DSL example: history-aware
- 20:39Consequences of representational equivalence
- 23:53SPAM FIGHTING
- 24:50Solution: Undo
- 26:28Summary
- 26:58The future!
related talks
25mProtecting Pinner Passwords
23mFighting phishing and spam with online machine learning on data streams
01h 00mRock ‘Em Sock ‘Em Robots: Bot Swatting Like the Pros
10mUsing Content Security Policy for Privacy
29mSocial Media Botnet Detection with Linkage Analysis and Machine Learning
52mDevil in the Haystack
43mRspamd — A High-Performance Spam Filtering System
45mPanel: From Spam to Speech: Policing the Next Generation of "Unwanted Traffic"