Apache Kafka and ksqlDB in Action : Let’s Build a Streaming Data Pipeline

on YouTube ↗29m

" summary

A working pipeline demonstrates Apache Kafka + ksqlDB taking a datagen stream of ratings, filtering out test devices, creating a ratings_live topic, pulling user state via MySQL CDC into Kafka with Kafka Connect, joining events to customer records, and emitting an unhappy platinum customers topic. Connectors run, messages flow into Elasticsearch + Kibana dashboards, and the system shows live-enrichment, windowed aggregates and pull queries so the alerts and visuals update in real time.

AI-written summary · about

" chapters (6)

1. 0:00Start
2. 0:26Demo
3. 2:59Kafka Connect
4. 11:23Integrating Databases in Kafka
5. 14:32Create a Table
6. 25:45Pull Query

" related talks

• 41mApache Kafka and ksqlDB in Action: Let’s Build a Streaming Data Pipeline!Robin Moffatt · All Things Open 2020
• 49mLet’s Build a Streaming Data Pipeline!Robin Moffatt · DataXDay 2020
• 26mConnecting distributed data with Apache KafkaJacob Bogie, TJ Laher · Azure Cosmos DB Conf 2022
• 57mTaming IoT Data: Making Sense of Sensors with SQL StreamingHans-Peter Grahsl · Voxxed Days Zurich 2019
• 08mPREVIEW: Kafka Stream Processing for Everyone with KSQLNick Dearden · Kafka Summit San Francisco 2017
• 01h 03mApache Kafka Event streaming platform for .NET developersViktor Gamov · NDC Sydney 2019
• 35mApache Kafka as a Monitoring Data PipelineJakub Scholz · DevConf.CZ 2021
• 18mBuild a data pipeline in 20 minutes with Kafka ConnectJocelyn Drean · DevFest Nantes 2018