Predicting Retention on the BBC Sounds Service Using Historic User Dynamics
summary
BBC Sounds uses temporal features and static demographics to score the probability each user returns next week, training on 100,000 users with 140 days of context and exposing a ~5 million weekly audience (5 million users). After PCA (sweet spot ~70–75% variance) and Bayesian hyperparameter search, simple linear models win: logistic regression matches XGBoost and outperforms RNN/CNN, so a cheap interpretable weekly probability feed is deployed for push and recommendation interventions.
AI-written summary · about
chapters (24)
- 0:00Retention prediction at BBC Sounds
- 1:10Modeling retention for BBC Sounds
- 3:59BBC Sounds product overview
- 6:24Defining retention and churn
- 8:07Data challenges in free audio
- 9:13Preprocessing for retention modeling
- 11:00Retention models and research
- 12:05Cost-effective model motivations
- 13:09Hyperparameter tuning insights
- 15:16Optimal temporal model configs
- 15:58Data recency impact on models
- 16:41Temporal vs static learning
- 17:42Model accuracy and deployment
- 19:07Why temporal models failed
- 20:31Next steps and improvements
- 21:34Retention prediction limits
- 22:17Churn modeling advice
- 22:39Q&A
- 23:01Data recency effects clarified
- 24:25Retention labeling technique
- 24:47Static features and ethics
- 25:53Deployment strategy for predictions
- 26:59Delivering model insights
- 28:22Closing remarks
related talks
38mUsing Recurrent Neuronal Networks for Customer Predictions
33mHidden Biases In Data: Moving Beyond Black Box Models
35mUsing Survival Analysis to understand customer retention
37mBayesian forgetting and Random Effects for forecasting TV ratings
30mDetecting Hidden Bias in Datasets Before Models Fail
24mUnlocking revenue: newcomer classification for boosted profits
13mBeyond Surveys: Real-Time User Satisfaction Prediction
01h 00mAI Powered Uplift Modeling and Stream Forecasting in Music Industry