Don't Let Users Find Your Outages: Synthetic Monitoring for Kubernetes Platforms
summary
Kuberhealthy runs recurring check pods every five minutes that create resources (Argo CD deploys a demo app, ingress + Route 53 record, then the check queries the API) and report success to Prometheus, catching failures metrics miss like Route 53 rate limits, cert-manager issuer outages, out‑of‑IP node scaling, or external‑secrets engine errors. Started a year ago across every cluster, synthetics drastically reduce mean time to discovery but require tuning to avoid observer‑effect load spikes and unexpected logging costs, and enable automated remediation possibilities.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 1:05Why synthetic monitoring for Kubernetes
- 3:36What synthetic monitoring measures
- 5:23Who benefits
- 6:27Metrics vs logs vs synthetics
- 7:08Simple synthetic check: external DNS
- 7:50Complex synthetic check: app deployment
- 9:14When to use synthetics in Kubernetes
- 11:44How synthetics work technically
- 13:11Pitfalls with synthetics
- 15:17Future plans and automation
- 16:41Key takeaway and Q&A session
related talks
25mDon’t Catch Feelings, Catch Issues With Kuberhealthy
30mOperating within Normal Parameters: Monitoring Kubernetes
12mKeynote: From Outage To Observability: Lessons From a Kubernetes...
42mMaking Resilient, Self-Adaptive, And Autonomous Kubernetes Clusters
19mKeynote: Monitoring Kubernetes Clusters with Prometheus
32mSurviving From Endless Issues Coming From 7K+ Kubernetes Clusters
35mLessons Learned Building a Global Synthetic Monitoring System
36m101 Ways to “Break and Recover” Kubernetes Cluster