Expedia Group's Downtime-Free Shift: Transitioning from Instance to IP...
summary
Expedia migrates NLBs from instance to IP mode using AWS Load Balancer Controller and Pod Readiness Gate, provisioning a temporary IP-mode NLB, switching traffic via ExternalDNS annotations, then converting original NLBs and migrating back; rollout covers ~180 production clusters and ~2,100 services with phased PCI-category pilots, 2‑day and 1‑week soak windows. Pilot fixes include an NLB target‑limit workaround and lowering client DNS TTLs; post‑migration timeouts and 5xx errors fall to near zero.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 0:33Meet Isha Shreya
- 1:30Agenda
- 2:54Why Does This Change Stand Out
- 3:59Road To Zero Downtime
- 8:05Solution Phases
- 8:45Root Cause Analysis
- 13:38Client Side Issues
- 14:10Solution Design
- 15:56Rollout Preparation
- 16:53Rollout Overview
- 19:10Rollout Approach
- 20:32Testimonials
- 21:17Conclusion
related talks
25mSwitching From External Load Balancing to consul & ingress [I]
42mCo-Evolution of Kubernetes and GCP Networking
35mLiving with the Pathology of the Cloud: How AWS Runs Lots of Clusters
32mCloud Native Global Load Balancer for Kubernetes
41mA Future Journey: How to Migrate 100 Clusters Between Clouds Without Downtime?
38mMigrating From Single-Node Kubernetes Control Plane To HA In Product...
13mBuilding Reliable Infrastructure for LLM Workloads on K8s
26mScaling Multi-cluster Kubernetes at Teleport Cloud