Kafka: The Must Know Pub/Sub
summary
Explains Kafka as a distributed pub/sub used as a microservice data backplane, covering components, CLI tools (kafka-console-producer/consumer, kafka-topics), Java producer/consumer code, serializers/deserializers, and murmur2 hash → partition sharding with an example of 25 partitions mapping Oracle to partition 19. Covers acks (0/1/all), retries, default retention of 168 hours, replication factor (RF=10 tolerates nine failures), consumer groups, rebalances, auto-commit every 5s vs manual commit, compaction, Kafka Connect, Kafka Streams, Spark and TensorFlow integration.
AI-written summary · about
chapters (10)
- 0:00Introduction to Kafka
- 3:30What Kafka is
- 10:39Kafka producers explained
- 22:25Programming Kafka producers
- 33:12Kafka consumers and groups
- 39:15Consumer group demo
- 44:32Kafka visualization tools
- 49:30Consumer rebalance and commits
- 50:12Retention and compaction
- 53:48Kafka in modern architecture
related talks
41mKafka as a Platform: the Ecosystem from the Ground Up
54mKafka as a Platform: the Ecosystem from the Ground Up
39mKafka as a Platform: The Ecosystem from the Ground Up
41mKafka as a Platform: The Ecosystem from the Ground Up
16mIntroducing Apache Kafka
36mApache Kafka Simply Explained
01h 02mKafka for .NET Developers
01h 05mKafka for .NET Developers