Every ML Model Deserves To Be A Full Micro-service
summary
Claim: every model ships as a real microservice with tracing, versioning and promotion. Implementation uses meta flow to store artifacts in S3, a lightweight Go control plane plus Python hosted functions on a FaaS/gateway, integrated with Spinnaker/Titus. Result: data scientists self-deploy and promote versions, six infra engineers support 100+ teams, deployments are reproducible, logged and load-balanced, and the system actually runs at Netflix production scale.
AI-written summary · about
chapters (11)
- 0:00ML model deployment at Netflix
- 2:05Real-time model use cases
- 6:18Engineers or data scientists
- 9:31Challenges in deployment
- 11:18Simplifying with Metaflow
- 13:26Metaflow and how it works
- 18:02Hosting with Metaflow artifacts
- 20:06Model versioning and promotion
- 23:16Evaluating deployment solutions
- 27:17Hosting system details
- 30:31Conclusion and future
related talks
21mHarnessing Microservices for Scalable, Fault-Tolerant ML Systems
26mMicroservices
47mIt is not just Microservices
24mFrom Model to Microservice - Machine Learning at Scale
20mAccelerating innovation using tiered AI Architecture
45mMicroservices: A retrospective
48mChallenges in Implementing MicroServices
33mHow you really get your data science models into production the cool way!