Architecture of an NLP Deployment
summary
Maps the pieces you actually need to run NLP in production and the decisions that matter: data ingestion and versioning, featureization, expensive distributed training, and isolated serving so training never starves request latency. Shows concrete tools and wins — GitOps with Flux to auto-deploy from git, container builder → registry → cluster flow, a Duolingo rearchitecture that cuts latency 750 to 14 milliseconds, and a portable stack using Kubeflow TFJob and Seldon plus JupyterHub for experimentation. Emphasizes autoscaling, sandboxed resources, auditing and continuous delivery.
AI-written summary · about
chapters (26)
- 0:00Introduction
- 1:13Agenda
- 1:51Define your problem
- 3:08History of architectures
- 5:52Examples
- 7:27NLP Microservices
- 9:50NLP Components
- 13:30Lessons Learned
- 13:46Common denominators
- 21:17Duolingo
- 24:34GCP
- 25:10AWS
- 30:53Optimization
- 32:05Empowering
- 32:53CoopFlow
- 34:34CoopFlow Concepts
- 36:56Kubernetes
- 37:09CaseOnit
- 38:01CaseOnit Example
- 39:08Jupiter Hub
- 40:00Serving Models
- 40:54Seldon IO
- 41:40CADA
- 43:13The Future
- 43:31Encouragement
- 43:56Questions
related talks
36mDataOps and MLOps: NLP Development and Deployment in Kubernetes made easy
39mOperating Deep Learning Pipelines Anywhere Using Kubeflow
21mMastering the DevOps Machine Learning Pipeline
57mDevOps for Machine Learning
33mBuilding Resilient (ML) Pipelines for MLOps
42mDeep learning beyond the learning
22mProductive Machine Learning and Deep Learning Projects
51mSpeeding up deployment of machine learning models