Live Q&A Session #2 – Shashi Gandham
summary
Moving storage into the GPU fabric creates concrete issues: implementing GPU Direct Storage to bypass CPUs, coordinating RDMA paths, and upgrading heterogeneous NICs to match GPU bandwidth; training produces many tiny, latency‑sensitive flows while checkpoints create few huge elephants, so ECMP fails without flow splitting or app-level subflows and requires QoS. Cloud virtualization must handle millions of endpoints, cryptographic isolation per task IP, and rolling upgrades across mixed NIC firmware; NIC bonding adds CPU/soft‑IRQ overhead, so a BPF-based multi-NIC approach is preferred.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:50Latency sensitive vs tolerant traffic
- 3:18Network virtualization in GPU clouds
- 5:02Multi-NIC vs NIC bonding
- 6:04Hotless upgrades of ASAPD demons
- 7:05Handling elephant flows with ECMP
- 8:59Fabric link failures during checkpointing
- 10:03ML workloads driving network innovation
- 11:52Public cloud workload constraints
- 12:12Announcing task IPs in multi-tenant setups
- 13:17Closing
