Keeping Redditors safe in real-time with Flink Stateful Functions
summary
Reddit builds a real-time safety platform called Snooron that uses Flink Stateful Functions with Kafka and Kubernetes to detect and act on harmful content at ingestion speed; it processes ~114 new posts per second (≈3.6B/year) and automates actions for ~7.4 removals/sec. Workers run as isolated RPC services (currently Python workers), state lives in RocksDB via Flink, messages convert JSON→Protobuf, deployments support multi-tenant scaling, replayable testing, and end-to-end load validation — and the platform is already in production use.
AI-written summary · about
chapters (12)
- 0:00Intro
- 3:57The Problem
- 5:24The Workflow
- 6:36Why Flink
- 9:00Why Snooron
- 10:33Example
- 12:55Data ingestion
- 15:13Workers
- 17:10Bootstrapping
- 21:13Worker deployment
- 27:18Lessons learned
- 29:23Summary
related talks
15mRealtime bot detection with Flink
25mReal-Time Fraud Detection at Scale
43mLarge Scale User Behavior Analytics by Flink
16mAnalysing Real time Activity Streams for Security
18mBuilding a Scalable, Resilient System for Real-Time Fraud Detection
13mBuilding Financial Fortresses: Real-Time Fraud Detection
29mBuilding Resilient AI Platforms: Financial Fraud Detection
15mSystematising Production Operations: Reddit Control Tower