Enhancing User Engagement Through Personalized Recommendation at Dow Jones

on YouTube ↗35m

" summary

Dow Jones builds an in-house NLP recommendation pipeline using article embeddings (JCO and comparison to Gemini), Lambda-triggered embedding generation, a vector store plus cache, and hierarchical clustering to populate homepage sections; it serves 10–50k users with 500–2,500 articles/day, replaces a costly vendor and yields 22% uplift in CTR on rollout; personalization uses linear combination of article embeddings for user vectors, a hierarchical clustering step to balance similarity and diversity, and a zero-shot classifier to filter daily/contradictory market pieces (2.5% error in a 100-article sample).

AI-written summary · about

" chapters (20)

1. 0:00Introduction to Dow Jones project
2. 1:10Company context and products
3. 3:39Challenge for premium customers
4. 4:42Wall Street Journal Plus creation
5. 5:48Balancing similarity and diversity
6. 7:11Embeddings for content representation
7. 10:02Embedding selection and optimization
8. 12:11Text chunking and embedding generation
9. 13:13Cosine similarity explained
10. 15:19User reading history approach
11. 16:20Hierarchical clustering for diversity
12. 17:45System architecture overview
13. 20:14Homepage clustering implementation
14. 22:02Production results and uplift
15. 22:45Linear combination of embeddings
16. 24:34A/B testing and cost savings
17. 25:18Detecting contradicting information
18. 26:42Zero-shot classifier for relevance
19. 29:15Evergreen articles and future plans
20. 30:18Q&A session

" related talks

• 27mEnhancing Personalization Using Recommendation Systems & Multi-Armed BanditsDmitriy M. · DSC Europe 24
• 28mThe Power of Personalized Customer Experiences Through AI InnovationsN. Trajkov · DSC Europe 23
• 01h 21mUnlocking Personalization: A Deep Dive into Modern Recommendation AlgorithmsSarang Gupta · DSC Europe 24
• 42mDocument Vectors in the WildJames Dreiss · Strata Data Conference 2018
• 22mAI-Powered Personalized Discovery with Graph Search and VLM-Enhanced EmbeddingsBrianna Lynn King, Hsin-Yi Lin, Anwaar Msehli, Yasel Garces · Optimized AI Conference 2026
• 27mTransform online shopping experiences with personalized recommendationsSarah Strobhar · AWS Machine Learning Summit 2021
• 39mMachine Learning for Relevance and SerendipityJenny Finkel · Strange Loop 2013
• 03mServing Billions of Personalized News Feeds with AIMeihong Wang · Artificial Intelligence Conference 2018 New York