How not to burn out your monitoring team
summary
Argues concrete fixes: mandatory half-day off after a sleepless production incident, weekly duck time to fix small nagging issues, and wider rotation or follow the Sun staffing. Calls for alerting that wakes only for real business‑breaking failures (e.g. things killing service within ten minutes) and using PagerDuty‑style routing to wake the right person. Advises logging only actionable data (no success noise or ephemeral storage), measures like Netflix "starts per second", Auto Scaling groups and Facebook‑style remediation scripts.
AI-written summary · about
chapters (9)
- 0:00Intro
- 0:55Gil Zellner
- 1:32Gils background
- 2:29How we got here
- 3:31The easy stuff
- 5:52The alerting policy
- 7:04Oncall people
- 7:39Logging
- 9:38Measuring
related talks
19mSensory Friendly Monitoring: Keeping the Noise Down
19mSensory Friendly Monitoring–Keep the Noise Down
30mOn call does not have to suck
27mSensory Friendly Monitoring: Keeping the Noise Down
26mSensory Friendly Monitoring- Keeping the Noise Down
22mRaise a Production System:Making on-call human-scale w/ Observability & Trust
58mCultivating Production Excellence
19mSensory Friendly Monitoring