Kafka as a Platform: The Ecosystem from the Ground Up
summary
Explains Kafka as an event-centric platform with concrete mechanics: immutable append-only logs, topics partitioned for ordering guarantees, producers/consumers and consumer groups as the unit of scale, and replication across brokers for fault tolerance. Demonstrates a Docker Compose demo ingesting stock trade events, using ksqlDB to maintain 15-minute windowed aggregates, pushing results to Postgres via Kafka Connect, and managing schemas with Schema Registry + Avro. Recommends planning partitions (example rule of thumb: start ~16) and shows retention vs compaction.
AI-written summary · about
chapters (11)
- 0:00Intro
- 1:22Events
- 5:45The log
- 7:53Topics
- 9:33Partitions
- 10:55PUB/SUB
- 22:04Brokers and replication
- 23:27Pretty good so far...
- 24:17Brokers and replication continued
- 35:07Demo
- 39:42Summary
related talks
39mKafka as a Platform: The Ecosystem from the Ground Up
01h 00mKafka as a Platform: the Ecosystem from the Ground Up
54mKafka as a Platform: the Ecosystem from the Ground Up
01h 00mKafka as a Platform: the Ecosystem from the Ground Up
41mKafka as a Platform: the Ecosystem from the Ground Up
01h 07mKafka Streams and KSQLDB
54mKafka: The Must Know Pub/Sub
53mDemystifying Kafka