Scalable Multi-Node AI Workloads in Multi-Tenant AI Clouds U...
summary
Built a production multi-tenant AI cloud from open source, using Open vSwitch (OVS) + Open Virtual Network (OVN) + OVN-Kubernetes CNI, with hardware TC/RTE flow offload to DPU/NICs so packet handling runs at wire speed and saves CPU. Design maps each GPU to its own NIC and one logical switch per rail (example: eight rails for eight NICs), uses RDMA over Ethernet (RoCE)/GPUDirect for low-latency all-reduce, claims near-100% fabric utilization versus ~60% with standard Ethernet, and runs in production for game streaming.
AI-written summary · about
chapters (7)
- 0:00Scalable multi-tenant AI clouds
- 3:30Hardware acceleration for AI
- 9:38OSS stack and GPU interconnects
- 15:45Pod networking with OVS/OVN
- 24:42Hardware offloading in Open vSwitch
- 31:29Network topology design
- 37:34AI data centers with open source
related talks
39mScalable Multi-Node AI Workloads in Multi-Tenant AI Clouds U...
01h 15mTutorial: Build, Operate, and Use a Multi-Tenant AI Cluster Based Entirely on Open Source
28mScaling AI Infrastructure
25mHow We Power the Largest AI Deployments on the Planet: Running Vir...
27mHarnessing Kubernetes for Scalable AI/ML Workloads
10mBuilding a Multi-Tenant AI Platform with the CNCF Ecosystem
25mHigh Performance Networking for Distributed DL Training in Production K8s
35mCase Study: AI-as-a-Service on Kubernetes at Scale and In Production