Case Study: AI-as-a-Service on Kubernetes at Scale and In Production
Talkat KubeCon + CloudNativeCon North America 2019 · on Wednesday, 2019-11-20· at 11:50· Room 6C - San Diego Convention Center, Upper Level·
summary
Case study of an AI‑as‑a‑service built on OpenShift + Kubernetes that runs massive ML pipelines with GPU scheduling and multi‑tenant namespaces, uses JupyterHub notebooks for self‑service, and deploys models via Open Data Hub operator and Seldon as REST services. Implementation details: Postgres metadata, RabbitMQ queueing, Spark for processing, Prometheus/Griffin for monitoring. Results claim a ~30% average model performance improvement and a ~70% increase in model productionization; work on GPU sharing and multi‑cluster scaling continues.
AI-written summary · about
chapters (23)
- 0:00Intro
- 1:03Who am I
- 2:12The Ministry of Defense
- 3:05Machine Learning
- 3:57Deep Learning
- 4:55DeepMind vs Good Champion
- 5:25Characteristics of Machine Learning
- 5:39Data Compute Intensive
- 9:39Top 3 Requirements
- 10:08What does a Data Scientist want
- 11:29What they did in steps
- 12:40Multiple GPUs
- 14:09Multitenancy
- 15:29Good Neighbors
- 17:39ML Expert
- 20:15Design Elements
- 21:18Challenges Next Steps
- 22:10Whats Next
- 22:20Collaborative Approach
- 23:48Open Data Hub
- 25:00Openshift
- 27:03Demo
- 33:50Summary
related talks
27mHarnessing Kubernetes for Scalable AI/ML Workloads
17mDevOps for Private AI: A Scalable Platform Using Kubeflow
27mGen-AI at Scale: Simplifying Orchestration of Healthcare Applications Across Multi-Cluster Environme
25mEmpowering AI-Driven Drug Discovery: Overcoming Challenges in Building a ML Platform on Kubernetes
49mKubernetes and MLOps for Scalable and Reproducible Generative AI
33mDeploying and Scaling Spark ML and Tensorflow AI Models
15mScaling AI-Driven Financial Services: Ethical ML at Scale
54mEconomics and Best Practices of Running AI/ML Workloads on Kubernetes