Machine Learning in Production: What to do after you’ve trained the models?
summary
Search is treated as a production case study: millions of unique queries times millions of customers times thousands of attributes makes realtime scoring impossible, so the system pre-computes pre-computed embeddings for customers and products, stores them in Dynamo, caches results in Redis, and composes predictions at query time via microservices to meet Tier 1 latency SLAs. Implementation uses protocol buffers for language-agnostic feature pipelines, experiments with SageMaker endpoints and PMML, and uses genetic algorithms to tune ranking coefficients.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 1:56How search works
- 8:54New world problems
- 16:00Protocol Buffers
- 17:37Tier 1 SLA
- 20:20Tier 2 Microservices
- 21:53Tier 3 Microservices
related talks
01h 05mMachine Learning Models and Productionization
29mMachine learning in production
31mMLOps In Practice – How To Run Your Machine Learning Models In Production At Enterprise Scale
21mBuilding Scalable ML Systems: Strategies and Insights
26mWhy Machine Learning Algorithms Fall Short...
19mDrive transformation through machine learning
47mMachine Learning in Production: An Intro to MLOps.
38mBuilding robust machine learning systems