Improving Reliability @ Meta: By Analyzing Historical Events That Led to SLO Violations
summary
Instagram and Meta move SLO tracking from siloed dashboards and spreadsheets into a unified tool called Sleek, add a structured annotations schema with hierarchical matchers in the CLI and UI, and use a collaborative annotation framework plus an automated investigator called Dr Pattinson automated annotations to populate root causes. Analysis shows short sub-second blips are the single biggest contributor to SLO loss, prompting projects that fix locality issues in certain data centers and suboptimal product queries, and an insights UI estimates percent impact per root cause.
AI-written summary · about
chapters (7)
- 0:00Reliability analysis at Instagram
- 3:36SLO tracking challenges
- 6:10Sleek limitations
- 9:02Annotating SLO violations
- 13:32Annotations across Meta
- 16:44SLO insights UI
- 18:50Annotation-driven reliability plans
related talks
18mSLICK: Driving SLO Culture
31mServerless Observability: a case study of SLOs
22mLook Ma, No Metrics: A Quest Towards Extracting SLOs
23mGoing from 30 to 30 Million SLOs
29mMeasuring Reliability: What Got Us Here Won't Get Us There
40mScaling SLOs with Kubernetes and Cloud Native Observability
43mOur Experience Tracking and Driving SLO Adoption at Goldman Sachs
37mReal World SLOs and SLIs: A Deep Dive