Virgo: Scale-out Data Center Network for AI

on YouTube ↗20m

" summary

Virgo is a dedicated accelerator-to-accelerator fabric: a flat two-layer non-blocking topology built from high-radix switches and skinny links (e.g., 200G 30s lanes) to collapse hops and speed deployment. Multi-planar fabrics give N× scaling and tolerate a plane failure (eight planes → 87.5% bandwidth remains for a 1.6T NIC example). It ships SRv6 source routing and switch-driven adaptive routing, supports C‑SIG in-band telemetry that aggregates per-hop metrics without header growth, and surfaces sub-millisecond bursts that 30s averages miss; deployed for Google Cloud A5x (NVIDIA Vera Rubin NVL72) and next-gen TPUs.

AI-written summary · about

" chapters (9)

1. 0:00Introduction
2. 1:08AI era demands
3. 3:16Virgo design principles
4. 6:26Virgo architecture
5. 8:59Multi-plane design
6. 13:13High-resolution telemetry
7. 15:40Demo: sub-millisecond insights
8. 17:26Virgo recap and scaling
9. 18:50Closing

" related talks

• 17mMeta’s Network Journey to Enable AIHany Morsy, Susana Contrera · Networking @Scale 2023
• 39mScalable Multi-Node AI Workloads in Multi-Tenant AI Clouds U...Girish Moodalbail, Leonid Grossman · CloudOpen at Open Source Summit Europe 2024
• 15mAquila: A unified, low-latency fabric for datacenter networksDan Gibson, Arjun Singh, Amin Vahdat, Ming Tan, Hema Hariharan, Eric Lance, Moray McLaren, Behnam Montazeri, Stephen Wang, Zhehua Wu, Sunghwan Yoo, Raghuraman Balasubramanian, Prashant Chandra, Michael Cutforth, Peter Cuy, David Decotigny, Rakesh Gautam, Alex Iriza, Milo M. K. Martin, Rick Roy, Zuowei Shen, Ye Tang, Monica Wong-Chan, Joe Zbiciak, Hassan M. G. Wassel · 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI '22)
• 29mLiving on the Edge with DCN NetworkingAnita Tragler, Bernard Cafarelli · OpenInfra Summit 2020
• 22mScaling agentic AI: From infrastructure to enterprise outcomes – Duncan NgDuncan Ng · PlatformCon 2026 Live Day New York
• 24mSilicon innovation and the modern networkPeter McKiernan, Brad Casemore · AWS Silicon Innovation Day 2023
• 19mBluebird: High-performance SDN for Bare-metal Cloud ServicesDeepak Bansal, Sarah McClure, Rishabh Tewari, Changhoon Kim, Manikandan Arumugam, Navdeep Bhatia, James Boerner, Simon Capper, Neeraj Motwani, Ranga Narasimhan, Urvish Panchal, Tommaso Pimpo, Ariff Premji, Pranjal Shrivastava · 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI '22)
• 19mScaling AI Network with DSFAnkur Singh, Ron He · @Scale: Networking 2025