Taming IoT Data: Making Sense of Sensors with SQL Streaming
summary
Demonstrates a working IoT pipeline: ingesting archived smart‑home power and weather traces via MQTT proxies into Apache Kafka, processing with KSQL (streams, tumbling windows, joins and aggregations), and persisting results with Kafka Connect into MongoDB using a MongoDB sink connector. Queries run as KSQL queries as files for production or interactive mode for exploration; stateful tables update continuously and feed a dashboard via MongoDB change streams and server‑sent events. Demo components run end‑to‑end and show live charts updating in near real time.
AI-written summary · about
chapters (8)
- 0:00Introduction to streaming
- 2:13Defining stream processing
- 5:49The KSQL stack
- 8:45Kafka foundations
- 13:05KSQL mechanics
- 18:55Operating KSQL
- 25:08Live demo: IoT sensors
- 45:17Integration and conclusion
related talks
31mFun with Sensors
20mSet your data streams in motion in the ocean!
20mBuilding realtime efficient queries in an IOT platform
35mPractical Pipelines: A Houseplant Alerting System with ksqlDB
34mBuilding and Scaling Robust Zero-code IoT Streaming Data Pipelines
43mHow Big Data is Powering the Internet of Things (IoT) Revolution
45mPractical Cloud Pipelines
34mStream processing IoT time series data with RabbitMQ & InfluxDB