How We Built a Complex SLO App Tightly Integrated with Grafana
summary
Grafana Cloud SLO app implements an Operator/Reconciler pattern that watches typed SLO objects and asynchronously creates recording rules, alerting rules, and summary dashboards; it auto-upgrades dashboards and alerts when the app ships new features. The team builds on an app platform with Q schemas and codegen to OpenAPI v3, a key‑value storage API with built‑in validation and admission hooks to parse PromQL, and a watch API so a small multi‑tenant operator fleet reconciles thousands of installs reliably.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 0:28Grafana SLO
- 2:30Measuring, responding and improving SLO
- 6:42Operator architecture
- 8:14How we use app platform
- 11:30Storage and validation
- 20:26Wrapping it all up
related talks
40mScaling SLOs with Kubernetes and Cloud Native Observability
23mGrafana SLO Demo: Prioritize critical resources with SLO-driven IRM
43mOur Experience Tracking and Driving SLO Adoption at Goldman Sachs
31mServerless Observability: a case study of SLOs
22mLook Ma, No Metrics: A Quest Towards Extracting SLOs
44mGrafana Labs Customers: What We've Learned Building Observability at Massive Scale
30mFrom Monitoring to Observability: Left Shift your SLOs with Chaos
25mAn SLO-Driven Approach to Enhance Kubernetes Cluster Reliability