AI/ML Networking Challenges: The Fast and the Finicky!
summary
Demoed network visibility using a probe‑mesh that measures one‑way delays at Q‑pair granularity, showing a healthy baseline at 1.47 Tbps aggregate with 100 Gbps NICs but slow flows at 43 µs (others ~2.6 µs) that drop throughput to ~1.38 Tbps. Simulated NIC flaps cause jobs to crash and force rollback to previous checkpoints; a resiliency shim reroutes bytes off congested paths and restores near‑line rates (~92 Gbps per NIC), trading brief degraded throughput for no job restart.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:47Audience roles
- 2:29AI workload layers
- 6:02Lack of visibility
- 7:26Measuring one-way delays
- 10:40Links that flap
- 12:27Simulating link flapping
- 14:53Congestion and throughput loss
- 18:47Networking pain points
- 19:08Key takeaways
- 19:51Q&A
related talks
40mOptimize Your AI Cloud Infrastructure: A Hardware Perspective
28mRock around the clock (synchronization
25mScaling Up, One Network Bottleneck at a Time
29mThe Cat and Mouse Game of Infra Cost vs. Latency
20mAnatomy of @Scale Machine Learning Network Operations
25mResQ: Enabling SLOs in Network Function Virtualization
21mConference wifi redux
28mDevOps Road Blocks