From Monitoring to Observability: Left Shift your SLOs with Chaos
summary
Shows a concrete workflow: instrument services with Prometheus + PromQL, define SLIs like container RSS or Docker pull remaining, and enforce SLOs as a quality gate with Captain in CI/CD so deployments fail when SLOs break. Demonstrates chaos experiments with Chaos Mesh (DNS failures) that reproduce a C++ memory leak in Kubernetes, observe rising RSS, and trigger Prometheus alerts. Includes practical numbers (SLO examples: zone serial age <1h, pull count >10) and working demos.
AI-written summary · about
chapters (10)
- 0:00Start
- 0:42Turn back time
- 2:13Golden Signals
- 3:44Devs debugging software
- 10:09Retrospective: SLI and SLO for Docker Limits
- 12:13Understand metrics sources
- 15:07Shift left with quality gates & SLO
- 18:33Your own Chaos
- 25:22Use case
- 28:52Recap
related talks
35mFrom Monitoring to Observability: Left Shift your SLOs with Chaos
31mServerless Observability: a case study of SLOs
36mConfidence with Chaos for Your Kubernetes Observability
40mScaling SLOs with Kubernetes and Cloud Native Observability
26mUnified Chaos Injection And SLO Validation Experiments In Kubernetes
37mUsing SLOs for Continuous Performance Optimizations of Your K8s Workloads
09mObservability in Kubernetes: A Signal- and SLI-Centered Approach to Reliability
31mSLX: An Extended SLO Framework to Expedite Incident Recovery