Building switches like servers: the strange world of overlapping networking
summary
Built custom switch hardware and separate software stack, running a standard CentOS image and treating the forwarding processes as services (thrift RPC) so fleet-style canary rollouts and automatic upgrades are possible; agent preserves ASIC state across reboots to avoid traffic interruption. Production timeline: ~1 year to prototype, deployed ~2.5 years ago, now shipping wedge 100 and fabric boxes backed by a Broadcom ASIC (≈1.6 Tb/s). Outcome: frequent 2–4 week agent updates, better visibility and automation, but increased operational coordination.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 0:19About me
- 1:25Overview
- 2:08Facebook Network
- 2:56Facebook Network Structure
- 4:31Data Center Network Structure
- 7:46How did we build this network
- 9:56Problems with this approach
- 14:21Our approach
- 16:00Hardware vs software
- 18:19How this worked in practice
- 19:55How this happened
- 20:55Software
- 25:09Services
- 27:42Learnings
- 30:51Next steps
related talks
41mHow to Build a Network Automation Stack From The Ground Up
15mHow Our SREs Safeguard Nanosecond Performance—at Scale—in an Environment Built to Fail
18mPast, Present, and Future of Network Operations
21mHardware Automation Deployment and Test with Big Scale in ChinaMobile NFV cloud
30mHow We Built an Automated Data Center Network in Less Than 6 Months
27mSWITCH: Operations, Data Management and Automation
50mEdge Computing: Appliance and Challanges
18mEngineering the Future: Alliander’s Journey Toward Software-Defined Substations