From €1M License to In-House Success: How We Built a Real-Time Recommendation System and Saved Millions Doing It
summary
Company replaces a €1M license by building a Kotlin real‑time recommender, deployed to 5% traffic then scaled to millions of requests per second, with an initial 48‑segment statistical model and daily retrained binary click predictor via Airflow that fails offline‑to‑online due to flaky render/click events (where healthy data shows 4% CTR uplift). Solution adds a Thompson sampling multi‑armed bandit layer on top, recovers lost clicks and conversions (avoids ~40% click / 44% conversion loss) and drives up to 20% lift for flex banners, saving over €1M per year.
AI-written summary · about
chapters (18)
- 0:00In-house recommendation journey
- 1:32Replacing €1M cloud tool
- 4:26Building from scratch
- 6:39Starting with simple model
- 8:06A/B test failure peak season
- 9:12Switching to ML batch
- 10:43ML model offline failure
- 11:47Marketing banner control
- 13:31Marketing requests true need
- 14:33Loss from ignoring real-time
- 16:19Solving with multi-armed bandits
- 18:31MAB vs random sampling
- 20:14Exploration vs exploitation
- 21:42Key lessons and future
- 23:13Q&A: Old system performance
- 24:15Q&A: Exploitation vs exploration
- 26:24Q&A: Classical vs MAB
- 28:13Q&A: Data quality issues
related talks
43mRecommendation Systems in Market Research
27mGenerating targeted AI revenue at mass scales
23mThe unreasonable effectiveness of data for better monetisation
32mResurrecting a Recommendations Platform
40mHow we built recommendations, cut the vendor, and made money
20mDataEngBytes 2023 - PER-T1-06 - Kamal Hossain
38mRecommendations at Rappi: from MVP to Data Product
27mEnhancing Personalization Using Recommendation Systems & Multi-Armed Bandits