🤖 Building a Telegram bot with Apache Kafka and ksqlDB

on YouTube ↗59m

" summary

Builds a Telegram bot that reads Bradford and Leeds car‑park CSV events into Apache Kafka, enriches and reserializes them with ksqlDB (protobuf schema in Schema Registry), and serves both materialised view lookups and push and pull queries over its REST API. Demo uses a 3‑minute source API, ingests every 30s, shows alerts (example threshold 1,000 spaces) and current states (examples: 73 full, 610 empty), streams to Elasticsearch via Kafka Connect, and the bot and alerts work end‑to‑end.

AI-written summary · about

" chapters (16)

1. 0:00Introduction
2. 3:10Demo
3. 9:57Telegram
4. 12:41API
5. 26:11State
6. 29:06State query
7. 38:06Using the same data
8. 39:51Kafka Connect
9. 41:36How we built it
10. 49:24Why use Kafka
11. 49:54Streams of events
12. 51:14ksqlDB
13. 54:04Running ksqlDB
14. 54:24Why Kafka
15. 55:09Wrap up
16. 56:14Questions

" related talks

• 41mApache Kafka and ksqlDB in Action: Let’s Build a Streaming Data Pipeline!Robin Moffatt · All Things Open 2020
• 29mApache Kafka and ksqlDB in Action : Let’s Build a Streaming Data PipelineRobin Moffatt · DevOps Pro Europe 2021
• 49mLet’s Build a Streaming Data Pipeline!Robin Moffatt · DataXDay 2020
• 57mTaming IoT Data: Making Sense of Sensors with SQL StreamingHans-Peter Grahsl · Voxxed Days Zurich 2019
• 01h 00mCalculating the Value of Pie: Real-Time Survey Analysis With Apache Kafka®Danica Fine · NDC Oslo 2023
• 31mFun with SensorsYogi Rampuria · Distributed SQL Summit Asia 2022
• 01h 03mApache Kafka Event streaming platform for .NET developersViktor Gamov · NDC Sydney 2019
• 39mKafka as a Platform: The Ecosystem from the Ground UpRobin Moffatt · GOTOpia Europe 2020