Multi-Cluster Observability: What you need to survive with a fleet
summary
Overview of a working multicluster stack using Prometheus, Thanos and Observatorium that centralizes metrics for telemetry, billing and fleet management. Uses Thanos receive architecture to remote write from thousands to ten thousand clusters, stores metrics up to three to four years, and handles 200 million active series scraped every 30s. Describes an operator to automate Thanos sizing and scaling, shows cost comparisons where DIY open source is materially cheaper at large scale.
AI-written summary · about
chapters (12)
- 0:00Introduction to multicluster observability
- 1:04Single cluster challenges and Thanos
- 3:32Single cluster monitoring setup
- 4:58Thanos architecture overview
- 7:47Multi-team monitoring with Thanos
- 9:32Multicluster observability setup
- 10:37Telemetry and analytics use case
- 13:09Billing use case with observability
- 14:16Fleet management with observability
- 17:51Introducing Thanos operator
- 20:58DIY vs vendor solutions comparison
- 23:10Q&A and closing remarks
related talks
29mMonitoring multiple Kubernetes Clusters with Thanos
36mRunning the Observability As a Service For Your Teams With Thanos
10mUpstream-First, High Scale Prometheus Ecosystem
31mHandling Billions of Metrics with Prometheus and Thanos
39mAbsorbing Thanos Infinite Powers for Multi-Cluster Telemetry
37mThanos: Prometheus in HA Mode with Persistent Storage
30mIntro to Thanos: Scale Your Prometheus Monitoring With Ease
24mObservability at Scale: How Open Systems Collects Telemetry from Over 10'000 Edge Devices Worldwide