An Engineer's Guide to a Good Night's Sleep
summary
Describes concrete operational practices used at FT and Skyscanner: migrated services onto Kubernetes to meet a 15-minute RTO, built active‑active US/UK stacks across ~180 microservices, use Kafka peaking at 2 million msgs/sec, implement automatic rollback in CI/CD, synthetic requests and structured event logs for probing production, idempotent processing, health/liveness probes, consumer restart-before-alert logic, chaos experiments and a Jenkins republish job so systems can limp to the next day instead of waking engineers at 3am.
AI-written summary · about
chapters (20)
- 0:00Intro
- 2:19"You need a mature operations team to manage lots of services, which are being redeployed regularly" Martin Fowler
- 2:53Empowered teams means the team also control the support
- 5:18Engineer's mindset
- 6:26Enable teams to own their own support models
- 7:52The team triages issues during the day
- 11:05Can our deployment times help this?
- 11:59Get your deployment system do automatic rollbacks
- 15:58Applications need to cope with change
- 20:52Automate failure recovery where possible
- 21:03Understand what your customers really care about
- 21:59You want to be the first to know about a critical failure
- 27:05Monolith to microservice timeline
- 27:17When can we release the chaos monkeys?
- 27:51Manual simulation of outages work too
- 28:57Multi region automatic system failovers
- 29:46Manual intervention should be simple
- 31:20At 3am just get the system to limp into office hours
- 31:52Break things and practice everything
- 32:35Understand what your customers care about?
related talks
32mA Day in the Life of a Kubernetes Engineer
19mThe Never-Ending Story of Site Reliability
42mStart Small, Scale Big: Building and Scaling Platforms and SRE Culture...
18mThe Good, the Bad and the Ugly: The 3 Learnings of an SRE
50mSafe Landings: Self-care In A High Intensity Technological World
32mMy Great Awakening: Top “Aha” Moments As Former Dev Manager
17mCreating Happy Engineers - Anecdotes from the Trenches
39mConfessions of a Systems Engineer: Learning from My 20+ Years of Failure
