Cloud-Native MLOps Platform for Python- and ML Engineers
Talkat Open Source Summit Europe 2023 · on Wednesday, 2023-09-20· at 16:50· Room 0D-2-0D-3 (Floor 0), Abandoibarra Etorb., 4, 48011 Bilbao, Bizkaia, Spain·
summary
Claims a cloud-native MLOps stack that takes a Python prototype to Kubernetes with minimal rework, centered on DocArray for multimodal documents and embeddings (example shows a 384-dim torch embedding) and a gRPC/HTTP-serving layer that supports dynamic batching, OpenTelemetry traces and metrics. Indexing plugins include in-memory HNSW, vv8 and Elasticsearch. A CLI/operator called jcloud operator adds autoscaling, GPU scheduling and DNS so flows deploy to cloud; demos show local POCs served remotely and the deployment workflow working end-to-end. Team is ~50 and project is ~3 years old.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 1:04Presentation outline
- 3:31Deep learning use cases
- 5:19ML deployment challenges
- 6:21Dockery for local POCs
- 9:08Dockery components
- 11:35Building POC with Dockery
- 15:28Serving models with Gina
- 17:56Orchestrating executors with Gina
- 19:24Adding observability with Gina
- 21:32Cloud deployment with Kubernetes
- 23:17Kubernetes for ML workloads
- 25:46Gina and Kubernetes integration
- 26:51Simplifying deployments with JCloud
- 27:54Summary
- 28:50Q&A
related talks
35mCloud-Native MLOps Platform for Python- and ML Engineers
36mDataOps and MLOps: NLP Development and Deployment in Kubernetes made easy
29mTeam Teardown: MLOps in the Cloud for IoT
22mBuilding a MLOps Platform with Kubeflow on GKE
32mBuild an MLOps platform on top of OpenStack
36mAccessible Neural Search and Cloud-Nativeness for ML Engineers
26mKubernetes and MLOps for Scalable and Reproducible Generative AI
31mProductionizing ML with ML Ops and Cloud AI