Building Zero-Instrumentation Observability
summary
Builds a Kubernetes-attached observability stack using fluent bit, open search and Prometheus, with Fluentbit as a DaemonSet, OpenSearch as distributed storage managed by an operator, and Prometheus + Thanos for long-term metrics. Claims zero-instrumentation (Hubble/ebpf + cAdvisor + kube-state-metrics), achieves 2–3 second log latency versus Azure’s 20s–3m, and reports ~70% cost reduction — about $1,800/month saved per data plane. Describes index/shard replicas, operator-driven upgrades, snapshot backups, and real-world fixes that made the system reliable.
AI-written summary · about
chapters (9)
- 0:00Introduction to the Observability Session
- 0:22The Need for an Observability Platform
- 1:55Initial Setup and Challenges
- 4:10Defining the Next-Gen Observability Stack
- 6:08Building the Log Management System
- 9:09Deep Dive into Log Management
- 23:30Metrics Management Overview
- 31:41Lessons Learned and Cost Savings
- 36:05Summary and Conclusion
related talks
40mBuilding a Scalable Observability Platform for Edge Compute Nodes
21mKubernetes Under the Magnifying Glass: Observability With OpenSearch
12mLightning Talk: Achieving Unified Observability for Cloud and Edge with FluentBit
28mBuilding Observability Pipeline with Fluent-Bit
01h 02mDesigning an open-source observability stack for Kubernetes
12mOpen-source observability is the way forward, and here’s why
41mResilient Observability at the Retail Edge: A Lightweight, Scalable, and...
41mSIG Instrumentation Deep Dive