Distributed Monitoring: How to understand the Chaos
summary
Explains how monitoring shifts from host-centric graphs to correlating metrics logs and traces, using InfluxData stack: InfluxDB + Chronograf + Kapacitor, per-node collection via per-node Telegraf sidecar and daemonset on Kubernetes, and tracing with OpenTracing -> Zipkin/Jaeger. Shows traces as span trees to find missing API calls, demonstrates tracing high-cardinality and cost (store for seven days then downsample), and argues alerting should target user-facing SLO failures rather than raw CPU spikes.
AI-written summary · about
chapters (15)
- 0:00Introduction to monitoring
- 2:55InfluxData monitoring stack
- 5:13Distributed system complexity
- 8:26Metrics, logs, and traces
- 12:47Understanding distributed tracing
- 15:35OpenTracing standard
- 18:35Trace data management
- 22:42Metrics collection methods
- 24:52Monitoring in Kubernetes
- 26:40Cultural shifts in operations
- 30:19Code instrumentation practices
- 32:35Focus on user experience
- 34:35Observability vs monitoring
- 37:14Debugging with traces
- 41:16Conclusion and Q&A
related talks
42mDistributed Monitoring: How to understand the Chaos
43mDistributed Monitoring: How to Understand the Chaos
38mInfrastructure Monitoring Basics with Telegraf, Grafana and InfluxDB
42mMonitoring Mesos with InfluxDB & the TICK Stack
23mHow to Monitor Application Performance in a Container-Based World
36mConfidence with Chaos for Your Kubernetes Observability
35mReal-Time Monitoring with Grafana, StatsD and InfluxDB
34mReduce Complexity with Distributing Tracing