Lightning Talk: Bringing Multi-Tenancy to GPU-Based Workloads
Lightningat Multi-TenancyCon Europe 2024 · on Tuesday, 2024-03-19· at 12:10· Pavilion 7 | Level 7.3 | Room S02·
summary
Argues that full tenant isolation on GPUs is essentially impossible with standard cgroups so practical options are dedicated GPU, MIG partitioning, or fragile time-slicing. Shows production use of the CNCF-sandbox framework capsu (plus kamaji) to enforce runtimeClass and nvidia.com/gpu resource quotas, letting providers pool GPUs and expose self‑service namespaces while the NVIDIA operator handles allocation. Describes a new katees GPU project that deploys virtual GPU-enabled clusters on shared shards and reports adopters already using it.
AI-written summary · about
related talks
05mLightning Talk: Sharing a GPU Among Multiple Containers
04mLightning Talk: Dynamic GPU Fraction and Sharing With Cloud Native Principle
34mDeclarative Device Virtualization: Orchestrating GPUs & Hardware in Cloud Native Environments
10mLightning Talk: K8s Issue #52757: Sharing GPUs Among Multiple Containers
28mShared Yet Isolated at Scale: Building Multi-Tenant Inference Platform on…
21mExperience Operating Large GPU Clusters at Organizational Scale
09m⚡ Lightning Talk: Advanced GPU-Orchestrated Workflows and HPC Integrations on K8s for Distributed AI/ML at Scale
38mImproving GPU Utilization using Kubernetes