Practical Pipelines A Houseplant Soil Alerting System with ksqlDB
summary
A Raspberry Pi reads capacitive I2C soil sensors (only four unique addresses) and a Python producer sends moisture and temperature events to Kafka every 5 seconds. Metadata uses Avro schemas in the Schema Registry; ksqlDB performs a stream‑to‑table join and a one‑hour tumbling window that emits an alert when 240 low readings occur (20 minutes of low values). A Kafka Connect HTTP sink posts messages to a Telegram bot, all hosted on Confluent Cloud and implemented with Raspberry Pi and ksqlDB.
AI-written summary · about
chapters (12)
- 0:00Project inspiration
- 5:34Kafka concepts
- 10:09Kafka architecture
- 15:20System design
- 18:34Hardware implementation
- 21:33Data schema definition
- 24:15Ingesting data
- 27:29Stream processing options
- 30:37Processing with ksqlDB
- 37:13Windowing and alerting
- 42:03Alerting with Telegram
- 45:55Resources and Q&A
related talks
28mPiping Rain in Python - Adventures in a minute latency Radar Pipeline
18mPipelines for Clinical AI: Real-Time Sepsis Detection
41mDeploying machine learning pipelines to production
43mTowards a full stack python monitoring+analysis framework
33mKafka streams at scale
45mResilient Predictive Data Pipelines
28mTackling Kafka, with a Small Team
32mReal-Time Data Streaming with Azure Stream Analytics