X1 Recommendations Platform: Started from the bottom..
summary
Comcast rebuilds X1 recommendations by replacing brittle HDFS + daily copies and Lambda handlers with a fleet of Flink jobs consuming Kafka/Kinesis and aggregating to S3, using Redis for low-latency caching and Couchbase for some state; a complex MapReduce chain that produced a weighted “because you watched” seed becomes a two-step external-API call, and an embedded personalization JAR is decoupled into a proxy web service. A/B tests on the new For You homepage and TV rows show measurable increases in movie and TV consumption.
AI-written summary · about
chapters (15)
- 0:00Introduction and background
- 1:27Case study overview
- 3:15Architecture and engineering focus
- 6:04Current personalized experience
- 6:24Past struggles with personalization
- 7:50How we improved the system
- 8:11Technical debt in ML systems
- 9:36Data platform reliability issues
- 11:41Moving from Lambdas to Flink
- 12:44Over-engineering pitfalls
- 14:50Simplifying the solution
- 15:53Importance of visibility
- 16:36Decoupling dependencies
- 17:39Architecture trade-offs
- 18:21A/B testing as a success driver
related talks
39mRecommendations at XING: From theory to production
32mBuilding an recommendation system for IPTV
32mResurrecting a Recommendations Platform
42mBuilding a Unified Viewing Experience with Multi-Provider TV Recommendation
45mReal-time, Streaming Advanced Analytics....
23mML-Powered Search & Recommendation 101
38mRecommendations at Rappi: from MVP to Data Product
41mRecommendations for Building Machine Learning Software