DevOps Transformation: Introducing Incident Management and Maximizing Monitoring Value
summary
Introduces incident management across over 40 teams and 120 services, builds centralized alerting with Grafana + Prometheus + Loki + Tempo and an alert manager that does automatic routing and 15/30 minute escalation. Delivers short, table‑style runbooks linked to dashboards, enforces only high/critical alerts out of hours, and runs onboarding, legal checks and on‑call rotations for ~300 developers. Claims faster MTTR, fewer missed incidents (stops inboxes of 300+ daily alerts), clearer RCA practice and incremental improvements; tooling and runbooks are already in use and shown to reduce outage impact.
AI-written summary · about
chapters (28)
- 0:00Introduction
- 0:38Meet Lena
- 1:14Agenda
- 2:11DevOps
- 4:45DevOps Loop
- 8:34Incident Management
- 10:18Practical Questions
- 17:27Alerting
- 18:21Monitoring
- 18:56Alert Management
- 19:55Rooting
- 20:57Escalation
- 22:17When escalation is activated
- 23:10Reaching out
- 24:04What works best
- 25:07Use meaningful alerts
- 26:25Prior prioritize alerts
- 26:51Always check alerts
- 27:48Runbooks
- 33:27Runbook Example
- 35:03Proof Read
- 35:13Postmortem
- 36:12On Call
- 37:11Starting on Call
- 38:25Trainings
- 39:16Knowledge Sharing
- 40:24What do we gain
- 42:00QA
related talks
05mDevOps and incident management: A recipe for success
30mKeeping Your DevOps Transformation from Crushing Your Ops Capacity
45mIncident Management - Talk the Talk, Walk the Walk
01h 04mIncident Management - Talk the Talk, Walk the Walk
45m200 Teams, 1 Goal: DevOps Maturity at Scale
05mWhy DevOps is more than just automation
37mIntroducing Reliability Toolkit: Easy-to-Use Monitoring and Alerting
48mIncident Management - Talk the Talk, Walk the Walk