Cruise’s Self-Driving Networking Journey
Talkat KubeCon + CloudNativeCon North America 2019 · on Wednesday, 2019-11-20· at 16:25· Room 11AB - San Diego Convention Center, Upper Level·
summary
Cruise scales Kubernetes across GCP regions to an average of 7k pods and solves cross-VPC routing by replacing point-to-point VPNs with interconnects with BGP for 100Gbps links. A surprise: Google internal load balancers only health-check ~50 nodes, which causes outages; the fix is to run an externalized nginx ingress pool managed outside clusters and autoscaled by CPU. Automation includes Jerry for IP allocation, CoreDNS + Route53 sync, RunScope tests for ingress, Stackdriver logs/metrics, and a Kol admission webhook; network policies remain disabled.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 7:50Creating more Clusters
- 10:18Service ingestion
- 15:48DNS story
- 19:40Common DNS structure
- 22:10Ingress monitoring
- 24:39Lesson Learned
- 26:03Security
- 27:00Kol
- 28:08Challenges
- 28:43Visibility
- 29:34Other issues
related talks
42mCo-Evolution of Kubernetes and GCP Networking
43mCloud Native - effective computing at scale
32mService Networking Within Air-Gapped Environments: Deployment Strategies and Operat...
34mEmbracing Upstream Kubernetes in Web Scale Organization
22mCloud Native Multi Cluster/Multicloud Global Load Balancer for Kubernetes
16mRunning Services at Google Scale
32mCloud Native Global Load Balancer for Kubernetes
29mAccelerate AppDev with KIK (Kubernetes-Istio-Knative)