Building a Multi-Tenant AI Platform with the CNCF Ecosystem
summary
Platform runs multi-tenant AI for internal and external researchers across GPUs and MNcore, balancing cost and isolation with a hybrid model combining namespace isolation and dedicated nodes. Isolation is layered: fork of HNC for hierarchical tenants, network policies, admission control and controller scope limits. Fairness uses queued compute quotas and per-tenant object limits, efficiency uses tight packing to avoid fragmentation and gang scheduling implemented as a custom scheduler plugin. System is deployed and operational for research workloads.
AI-written summary · about
related talks
39mScalable Multi-Node AI Workloads in Multi-Tenant AI Clouds U...- 39mScalable Multi-Node AI Workloads in Multi-Tenant AI Clouds U...
09mLightning Talk: Bringing Multi-Tenancy to GPU-Based Workloads
01h 15mTutorial: Build, Operate, and Use a Multi-Tenant AI Cluster Based Entirely on Open Source
47mBuilding Trust In The AI Era
36mPanel: Control Plane vs Data Plane: Untangling the Tenets of Multitenancy
39mDesigning & Securing Multi-Tenant Runtime Environment at the New York Times
21mExperience Operating Large GPU Clusters at Organizational Scale