Empower Heterogeneous Edge AI Acceleration with K8s
Talkat Kubernetes on Edge Day Europe 2022 · on Tuesday, 2022-05-17· at 11:00· Pavilion 4, Room C | Level 2 | Central Forum·
summary
Kubernetes extension detects PCI/USB IDs with Node Feature Discovery, auto-labels devices, and deploys vendor device plugins so scheduler places pods on matching nodes; an interposer auto-compiles models with backends (Apache TVM, Intel OpenVINO, NVIDIA TensorRT, remote CUDA) and caches artifacts so unmodified TensorFlow/PyTorch/ONNX code uses accelerators. Demo shows device plugins registering, pods scheduled, and TVM interposer giving up to 8× speedup on NVIDIA and 19× on AMD for MobileNet, with transparent model caching and remote GPU attach.
AI-written summary · about
chapters (6)
- 0:00Introduction and agenda
- 1:07Edge AI challenges
- 3:48Acceleration architecture
- 8:22Kubernetes deployment
- 16:56Demonstration
- 22:00Performance and future plans
related talks
21mUnlocking How To Efficiently, Flexibly, Manage and Schedule Seven AI Chips in Kubernetes
27mUnlocking Heterogeneous AI Infrastructure K8s Cluster: Leveraging the Power of HAMi
27mHarnessing Kubernetes for Scalable AI/ML Workloads
30mDesigning a Hybrid AI Platform on Kubernetes
25mMaking the Most Out of Your Hardware Accelerators in a Kubernetes Cluster
30mRunning AI Workloads in Containers and Kubernetes
37mDeep Learning Neural Network Acceleration at the Edge
06mEdge AI in Practice: Lessons from Producti...