How HashiCorp does Incident Management
summary
HashiCorp uses a central SRE-run incident program for HCP and hosted products with Sev1–Sev4 severities, PagerDuty and Blameless for coordination, and a documented role set (first responder, investigator, incident commander). A recent telemetry alert lasts ~40 minutes, traced to a misbehaving container causing DB errors that delayed a workflow engine and dropped monitoring; the incident is downgraded to Sev2. Process includes 3-business-day retros, weekly ops reviews, quarterly trend analysis, game days, and action-item triage. PagerDuty and Blameless
AI-written summary · about
chapters (13)
- 0:00Introduction and welcome
- 1:46Incident process origins
- 3:30HashiCorp Cloud Platform architecture
- 4:55Incident roles explained
- 7:44Severity levels and criteria
- 10:34Incident response workflow
- 13:44Real incident walkthrough
- 15:30Retrospective and blameless culture
- 17:37Weekly and quarterly reviews
- 20:49Future improvements planned
- 24:23Strengths and ongoing efforts
- 26:50Lessons learned and scaling
- 29:39Q&A and closing
related talks
35mHow HashiCorp SREs Built HCP's Incident Management Program
37mIncident management for small service teams
20mHow to Have an Operational Incident
27mManaging Incidents
13mNavigating SRE/Incident Management
49mData Driven Observability & Incident Response
33mDon't Panic! Effective Incident Response
15mSustainable Incident Management for happy SRE teams