Apache Kafka and ksqlDB in Action: Let’s Build a Streaming Data Pipeline!
summary
Live demo shows Apache Kafka plus Kafka Connect and ksqlDB moving and transforming real traffic: a ratings topic streams into Elasticsearch/Kibana (index grows from 3257 to 3262 docs), a Debezium MySQL connector captures customers, ksqlDB creates streams and tables, joins enrich ratings into a new topic (ratings_enriched), and an aggregate table updates (5,610 → 5,645 processed). Connectors report “running,” enriched topics materialize, push and pull queries return current state — the pipeline actually works end to end.
AI-written summary · about
chapters (10)
- 0:00Intro
- 1:28Demo Overview
- 3:00Using ksqlDB
- 5:07Using elasticsearch
- 9:05ksqlDB
- 10:58schema registry
- 21:23table vs stream
- 26:20Stream vs table
- 28:35Kafka topics
- 32:54Building an aggregate
related talks
29mApache Kafka and ksqlDB in Action : Let’s Build a Streaming Data Pipeline
49mLet’s Build a Streaming Data Pipeline!
08mPREVIEW: Kafka Stream Processing for Everyone with KSQL
26mConnecting distributed data with Apache Kafka
57mTaming IoT Data: Making Sense of Sensors with SQL Streaming
42mData Streaming for Microservices using Debezium
01h 03mApache Kafka Event streaming platform for .NET developers
39mKafka as a Platform: The Ecosystem from the Ground Up