Live Q&A Session #1 – James Zeng
summary
Shows that tensor parallelism on a single host is practical when high‑bandwidth links like MVLink or AMD XDMI exist, but moving to RDMA across hosts forces extra parallelism, careful load‑balancing and padding costs; concrete example: a 128‑expert MoE with batch 64 only loads ~64 experts. Describes tooling: symmetric memory in PyTorch, DDA and device‑initiated streams (MStream/Rockstream) and how kernel fusion and backend patches change latency. Claims autoscaling and tracing scale to 100k GPU scale with focused triggers and timeouts everywhere.
AI-written summary · about
chapters (8)
- 0:00Q&A session
- 0:24Model parallelism to hardware
- 3:55NIC failures during runtime
- 5:21Shared-memory technologies
- 8:53Scaling to 100K GPUs
- 10:42Direct Data Access and CNA
- 12:49Noisy neighbor effects
- 21:25Stable production
