Kafka Message Broker Under High Load
summary
Kafka is presented as a unified, high-throughput distributed message broker used to handle terabytes of logs from dozens of providers and to support both realtime and batch pipelines. The talk explains why the team adopted Kafka and how it was integrated into their pipelines, how they achieved exactly-once delivery, a real incident where a typo caused a multi‑fold load spike and what they learned from it, and which Kafka metrics to monitor to detect problems.
AI-written summary · about
chapters (4)
related talks
01h 31mDeveloping Real-Time Data Pipelines with Apache Kafka
53mDemystifying Kafka
05mIntroduction to Kafka Internals
01h 00mPutting Apache Kafka to Use
39mKafka as a Platform: The Ecosystem from the Ground Up
18m"Kafka's Metamorphosis"
41mKafka as a Platform: The Ecosystem from the Ground Up
05mPREVIEW: 1 Day, 1 Data Hub, 100 Billion Messages