Apache Kafka and KSQL in Action: Let’s Build a Streaming Data Pipeline!
summary
Live demo builds a full streaming pipeline that mirrors MySQL rows into Kafka, enriches review events with customer data, filters low-star reviews for platinum customers and pushes alerts to Slack and Elasticsearch. Uses Kafka Connect (Debezium/DBZ for CDC), the Schema Registry, and KSQL to create streams/tables and continuous queries. The demo shows topics created (poor_ratings), counts rising from ~68 to ~95 processed records and surfacing ~45 unhappy priority customers in real time.
AI-written summary · about
chapters (6)
- 0:00Start
- 1:47Kafka is a Streaming Platform
- 4:42A Dumb Pipeline
- 9:10Let's Build It!
- 25:02KSQL for Anomaly Detection
- 33:24The Stream Table Duality
related talks
53mApache Kafka and KSQL in Action : Lets Build a Streaming Data Pipeline!
01h 07mKafka Streams and KSQLDB
01h 00mIn Depth Kafka Streams
44mNo More Silos: Integrating Databases and Apache Kafka
03h 00mStream Processing, Going Beyond Pub/Sub with Kafka Streams and...
42mdistributed stream processing for everyone with KSQL #bbuzz
39mAnalysing Public Kafka Data from NASA Satellites
39mKafka as a Platform: The Ecosystem from the Ground Up