The Billion-Request Content Recommendation System Challenge
summary
Claim: a recommendation API that scales to ~1.3 billion pageviews per day and shifts heavy work to background to cut median latency from ~2s to milliseconds and max from ~27s to ~1s. Architecture uses webq worker queueing with RabbitMQ, a Redis cache called the Rex store, batch embeddings via sentence Transformer, and load testing with Vegeta. Built components include candidate refresh cron jobs and customizable URL params; results show high cache hit rates and substantial real traffic performance gains.
AI-written summary · about
chapters (10)
- 0:00Billion-request challenge intro
- 1:14Engineering over algorithms
- 2:17Content recommendation challenges
- 5:50Recommendations at Chartbeat
- 9:01WebQ worker architecture
- 12:15Client-side API efficiency
- 16:09Recommendation cache store
- 18:12Cache key identity problems
- 23:32Load testing engine
- 32:02Key takeaways
related talks
20mDataEngBytes 2023 - PER-T1-06 - Kamal Hossain
32mResurrecting a Recommendations Platform
23mML-Powered Search & Recommendation 101
39mRecommendations at XING: From theory to production
47mHow to deploy a recommendation service with a BERT-like model to high-load production
28mBuilding a Recommendation Engine for Bus Search Based on Real-Time Feedback of Users
30mFrom €1M License to In-House Success: How We Built a Real-Time Recommendation System and Saved Millions Doing It
29mBuilding Generative Content Understanding Systems