Kafka as a Platform: the Ecosystem from the Ground Up
summary
Overview of Kafka as a full streaming platform: shows concrete building blocks (topics, partitions, replication, consumer groups, broker replication) and two live demos using docker/kafkacat that produce/consume, rewind offsets and scale consumption; demonstrates integration with Kafka Connect to move data to Elasticsearch and stream processing with ksqlDB and Kafka Streams to filter, window and aggregate trades, using Avro plus schema registry for compatibility; retention examples (7 days, 10 years, forever) and JVM broker deployment details.
AI-written summary · about
chapters (20)
- 0:00Intro
- 1:24What are Events?
- 4:33A platform for events
- 5:11The Log
- 6:54Topics
- 8:53Partitions
- 9:37Publisher API
- 10:45Message keys
- 14:10Consumer API
- 19:20Consumer Groups
- 22:30Brokers
- 24:39Demo 01 - Kafka producer/consumer
- 30:30Kafka use cases
- 32:43Kafka Connect
- 36:40Schemas & Serialisation Formats
- 44:40Stream processing introduction
- 46:50Kafka Streams
- 48:03ksqlDB
- 50:16Demo 02
- 57:05Recap & Resources
related talks
41mKafka as a Platform: The Ecosystem from the Ground Up
39mKafka as a Platform: The Ecosystem from the Ground Up
29mKafka on Kubernetes: Does it really have to be the hard way?
02h 05mA Deep Dive into Apache Kafka This is Event Streaming
53mDemystifying Kafka
01h 03mApache Kafka Event streaming platform for .NET developers
33mKafka streams at scale
26mKafka in the Wild