CAP Theorem Reloaded
summary
Uses CAP as a practical framework for telemetry pipelines, claiming partition tolerance is non‑negotiable and you must trade between consistency (correctness) and availability (ingestability). Cites volume: ~1 TB/day for small orgs up to a few PB/day for large ones. Recommends OpenTelemetry Collector processors—probabilistic and tail-based sampling, memory limiter, and a shared‑state quota enforcer—to implement sampling, filtering, aggregation and tiering so collectors stay up while accepting some eventual inconsistency.
AI-written summary · about
chapters (9)
- 0:00Introduction and Speaker Background
- 1:04Understanding Telemetry Challenges
- 3:01Introduction to CAP Theorem
- 5:21Applying CAP Theorem to Telemetry
- 9:04Open Telemetry and Its Components
- 10:10Managing Logs and Metrics
- 12:55Handling Tracing and Sampling
- 15:56Quota Management in Telemetry
- 18:44Key Takeaways and Conclusion
related talks
52mCAP Theorem - two decades and few clouds later
01h 15mCAP Theorem — two decades and few clouds later
41mUnderstanding the CAP Theorem with Apache Kafka
57mScaling the CAP Theorem for Modern Distributed Systems
47mDatabase Resiliency
54mDay 1 Session 1
29mInside the Telemetry Data Plane: Constraints, Tradeoffs, and Scale
33mReal-World Sampling – Lessons Learned...