GPU as a Service Over K8s: Drive Productivity and Increase Utilization
Talkat KubeCon + CloudNativeCon North America 2019 · on Thursday, 2019-11-21· at 16:25· Room 17AB - San Diego Convention Center, Mezzanine Level·
summary
Demonstrates an open source stack that turns notebooks into elastic, GPU-backed functions: MLRun builds and runs jobs on Kubernetes, auto-packages code, attaches GPUs only while jobs run, and integrates with Kubeflow pipelines and KFServing. Shows RAPIDS/cuDF on an NVIDIA T4 doing a JSON aggregation in ~1.4s vs ~43s on Pandas (30x speedup (cuDF vs Pandas)), uses dynamic GPU attach to avoid idle cost, and an elevator pattern to fill GPUs boosting throughput roughly 4x.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 0:52Why do projects take time
- 2:34What are GPUs used for
- 4:05GPU as a Service
- 4:47Architecture
- 9:00Functions
- 10:29Serving
- 11:45ML Functions
- 15:50Demos
- 16:41Use Case
- 18:27Running Functions
- 19:45Running an MPI Job
- 21:30Deploying the Results
- 24:33daisychaining functions
- 25:47Job as a pipeline
- 30:27Questions
related talks
16mGPU-Accelerated Data Science for PyData Users
38mAll You Need to Know to Build Your GPU Machine Learning Cloud [B]
01h 08mGPU Data Science with Rapids
36mProduction GPU Cluster with K8s for AI and DL Workloads
33mAI workloads on Kubernetes - How to maximize GPU utilization and cut costs
01h 22mHigh-Performance Data Science at Scale with RAPIDS, Dask, and GPUs
01h 36mRAPIDS: GPU-Accelerated Data Science for PyData Users
32mGPU Machine Learning From Laptop to Cloud