ML Research in Zen: How to Live Without KPIs?
summary
Describes how the ML teams behind Yandex Zen handle research-driven recommendation development without relying on fixed KPI targets. Covers the platform’s recommendation systems and the challenges of running experimental ML research that can fail, and discusses organizational and engineering practices used to develop, evaluate and ship improvements despite uncertainty.
AI-written summary · about
chapters (9)
- 0:00ML at Zen introduction
- 1:07Simple vs research tasks
- 2:16Time spent as metric
- 4:04Recommendation systems R&D
- 5:53KPIs in research
- 7:19Global research lessons
- 8:22Expert oversight in R&D
- 9:28Research freedom and challenges
- 10:11Publishing findings
related talks
30mNothing to Recommend It: An Interactive ML Outage Fable
39mMachine Learning Engineering
48mPrinciples and Practices of Building Machine Learning Systems
41mRecommender Systems: Crafting Effective Personalised Experiences
15mHacking ML into Your Organization
43mRecommendation Systems in Market Research
30mBuilding Teams and Culture that Support ML Innovation
26mML Product Experiments at Scale