Practical Pipelines: A Houseplant Alerting System with ksqlDB
summary
Builds a live pipeline that reads soil moisture from Raspberry Pi + capacitive I2C sensors every 30 seconds, serializes readings as Avro into Kafka, enriches them with plant metadata written from a Telegram bot, and runs a ksqlDB tumbling 6‑hour window that emits an alert only if a plant has ≥120 low readings in that window. Uses Confluent Cloud (Kafka Connect, schema registry), a compact topic and non‑committing consumer for mappings, and an HTTP sink to send Telegram alerts.
AI-written summary · about
chapters (35)
- 0:00Intro
- 3:10The perfect opportunity
- 3:37What is Kafka
- 3:58Unpacking
- 5:30Events
- 6:23Immutability
- 11:15Building the Pipeline
- 11:27Cloudbased Kafka
- 12:19Raspberry Pi
- 13:39Data
- 16:04Serializing Producer
- 17:02Readings into Kafka
- 17:38Metadata
- 18:06Kafka
- 18:58Telegram
- 19:35Conversation Handler
- 19:47User Input
- 19:58Producer Input
- 20:10Adding Metadata
- 20:40Updating Metadata from Phone
- 22:38Stream Processing
- 22:49Kafka Options
- 23:48Using ksqlDB
- 24:29Create table statement
- 25:01Define string ID
- 25:16Readings
- 25:31Join
- 26:08emitChanges
- 26:26Push vs Pull
- 26:36The Goal
- 27:52Key Details
- 28:06Window
- 28:34MidFinal
- 31:08Summary
- 31:38Questions
related talks
52mPractical Pipelines: A Houseplant Alerting System with ksqlDB
45mPractical Cloud Pipelines
56mTaming IoT Data: Making Sense of Sensors with SQL Streaming
31mFun with Sensors
20mSet your data streams in motion in the ocean!
28mBuilding an IoT Monitoring App with InfluxDB
34mBuilding and Scaling Robust Zero-code IoT Streaming Data Pipelines
21mObservability Made Simple: Monitoring Plants with IoT & Grafana