How to Compute CTR for 100M Items in Real Time and Survive
summary
The talk explains how Odnoklassniki computes CTR for hundreds of millions of items in real time, handling millions of events per second and feeding those results into ranking that scores tens of millions of candidates per second. It describes the streaming-data technologies and platforms used (Java-based real-time/stream processing systems), practical engineering changes made to those components, and additional analytics tasks enabled by the streaming platform.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 11:19Real-time CTR problem
- 16:44CTR system architecture
- 26:34Apache Kafka role
- 33:40Samza task details
- 41:29System monitoring
- 46:07Streaming Index optimization
- 51:00Summary
related talks
08mReal-Time Data Systems: Speed, Scale, and Resilience
30mData Processing in RTB: Fast, Cheap and 98% Accurate
38mCrunching data in real time for Fortune500 clients
52mThe ML wedding between millions of products, or how to withstand load in a stream
30mFive Challenges in Real-time Stream Processing and Five Solutions
38mThe story of building a real-time streaming analytics system
28mBest of Stream + Batch: When Streaming Analytics Isn’t
44mProcess and analyze millions of events per second with open source tools