Turbocharging AI/ML workloads: Revving Up Speed and Resilience
summary
Demonstrates that lack of Q-pair visibility causes undiagnosed slow flows and can cut throughput by up to 50%, shows live Oracle Cloud demos measuring one‑way delays (600 ns same‑rack, 1.5 µs cross‑rack) and 1.4 Tb/s job throughput on 16×100 Gb NICs, reproduces a single NIC flap that crashes jobs and contrasts a software fix that provides automatic recovery from NIC flapping plus congestion‑aware load balancing to keep jobs running at ~92 Gb/s per NIC.
AI-written summary · about
chapters (14)
- 0:00AI workload challenges
- 1:26AI workload layers and networking
- 3:11Infiniband vs RoCEv2
- 4:14AI network profiling and all-reduce
- 6:21AI network challenges today
- 7:48Demo: fine-grained visibility with probe mesh
- 9:56Link flapping reliability issues
- 10:59Demo: simulating and recovering from link flapping
- 13:52Auto recovery from link flapping
- 14:15Predictable performance and congestion
- 14:58Demo: congestion-aware load balancing
- 16:41Key takeaways for AI networks
- 17:46AI network intelligence and unified circuits
- 18:51Future of AI networks
related talks
25mResilience for AI Workloads at Scale: The Fast and the Finicky!
19mAI/ML Networking Challenges: The Fast and the Finicky!
30mThe Gashlycrumb Tinies of AI Networking You Must Know (or Languish!)
01mThe Future of AI Networking Starts Here
07mWelcome Remarks
15mLive Q&A with Speakers – Joseph Provine
21mDesigning Scalable Networks for Large AI Clusters: Challenges and Key Insights
13mLive Q&A Session #2 – Shashi Gandham