Operationalizing Machine Learning
summary
Levi's goes from outsourced experiments to over a dozen models deployed and ROI positive in about two years, replacing a third‑party recommender that refreshed every 24 hours with an in‑house pipeline that cuts 10–12 second latency down to production performance. Pipeline uses Elasticsearch early, then AWS Step Functions, SageMaker and SageMaker Ground Truth for active labeling, and Dataiku for orchestration. Prototyping uses Wizard of Oz testing, prioritize with effort‑impact and Eisenhower matrices, and validate with A/B tests so models actually change business metrics.
AI-written summary · about
chapters (16)
- 0:00Introduction to ML transformation
- 3:30Building a solid data foundation
- 5:56Engineering beyond the black box
- 7:41Identifying the right problems
- 9:06Two classes of ML problems
- 11:11Prioritizing ML initiatives
- 13:38Building capabilities over projects
- 15:21Building data-driven products
- 17:48Prototyping and validation
- 18:50Case study: product recommendations
- 21:16Reducing code complexity
- 22:37Tools for ML workflows
- 24:21Lessons from training data
- 25:45Data-efficient methods
- 26:28Making data-driven decisions
- 28:13User-driven product decisions
related talks
19mDrive transformation through machine learning
32mAutomating Production Production Level Machine Learning on AWS
33mMachine Learning Engineering Done Right
43mMachine Learning Ops: DevOps for Data Science
44mPragmatic Machine Learning in the Cloud
37mStreamlining Machine Learning Operations
29mMachine learning in production
39mDecision Making Based on Machine Learning at Outfittery