Service Messy: Pagerduty's Service Mesh Migration
summary
Pagerduty migrates roughly 500 services to a service mesh using an away team model that makes initial code changes and asks teams to review; after a year migration hits 95% complete. Migration happens in two phases: replace bespoke console discovery with name.namespace.svc.cluster.local, then flip a Helm flag to enable Istio sidecars. Hard lessons: retries and canaries behave badly until both downstreams and upstreams are on-mesh (retries can multiply n^2), Envoy/Istio resource combinatorics and a monolithic Rails app cause timeouts and production-only failures.
AI-written summary · about
chapters (5)
- 0:00Introduction
- 1:02Before service mesh migration
- 15:30PagerDuty's migration
- 29:42Migration lessons
- 35:33Current status
related talks
56mService Meshes - Powering the next wave of microservice architectures
26mIstio Migration: 50k+ Pods, Minimal Impact, Lessons Learned
45mService Mesh War Stories - Lessons Learned the Hard Way
40mThe World after Microservice Migration w/ Service Mesh & Contract Testing
53mService Mesh: What, Why, And How?
39mGrow with Less Pains - Meshing From Monolith to Microservices
26mService Mesh Failure Stories at Scale
56mHow to get a grip on your microservices system using a service-mesh