Deep Dive: The Stream, the Cache, and the Store: The History and Development...
summary
Walks through three components: Loggregator moves UDP to gRPC to raise delivery reliability from about 75% toward production-grade numbers; a reverse log proxy adds selectors and buffering for ordering; Log Cache is an ephemeral, co‑located Doppler component that stores per‑source red‑black trees for 5–15 minute lookback, removing noisy‑neighbor interference; Metric Store forks Log Cache, backs metrics with an Influx-compatible TSDB, exposes the Prometheus API and query language and ships with 42 days default retention so persistent charting and SLO tracking work.
AI-written summary · about
chapters (7)
- 0:00Start
- 0:38Why are we here?
- 1:28Operational definitions.
- 5:52The Stream
- 7:12Loggregator What is it?
- 20:01Log Cache What is it?
- 28:51Metric Store
related talks
32mAddressing Delivery Reliability in Loggregator
40mObservability that scales: Analyze data in stream and query remotely
28mCortex 101: Horizontally Scalable Long Term Storage for Prometheus
27mObservability with Prometheus and beyond
28mFrom Fragmented Logs to Unified Insights: Scaling O11y
30mBuilding a Large-Scale, Streaming-Based Logging and Monitoring Solution
41mResilient Observability at the Retail Edge: A Lightweight, Scalable, and...
31mZero to data infrastructure