Kafka as a Platform: the Ecosystem from the Ground Up
summary
Shows how an event as a key/value byte becomes a scalable platform: topics split into partitions with ordering and configurable retention, producers/consumers that rewind offsets, and brokers with replication for fault tolerance. Demonstrates integration via kafka connect (Docker demo using kafkacat), schema-backed serialization with Avro and a schema registry, stream processing with Kafka Streams or ksqlDB (15-minute tumbling-window trade counts) and sinking aggregates to Elasticsearch. Explains tradeoffs, partitioning keys, and when to use these Confluent components.
AI-written summary · about
chapters (17)
- 0:00Intro
- 1:24Events
- 6:53Topics
- 8:53Partitions
- 9:38PubSub
- 14:07Producing Data
- 14:32Reading Messages
- 18:27Kafka Consumer
- 22:29Kafka Broker
- 24:37Kafka Broker Demo
- 30:23Whats Next
- 33:43Kafka Connect
- 37:33Asynchronous coupling
- 42:49Schemas
- 50:05Demo
- 57:02Recap
- 58:05Resources
related talks
41mKafka as a Platform: The Ecosystem from the Ground Up
39mKafka as a Platform: The Ecosystem from the Ground Up
29mKafka on Kubernetes: Does it really have to be the hard way?
53mDemystifying Kafka
26mKafka in the Wild
20mKeynote: Apache Kafka and the Rise of the Streaming Platform
01h 03mApache Kafka Event streaming platform for .NET developers
02h 05mA Deep Dive into Apache Kafka This is Event Streaming