Internet of Harvesting Powered by Apache Ignite

on YouTube ↗43m

" summary

Describes an architecture for collecting and analyzing large heterogeneous device telemetry from agricultural machinery made by Rostselmash, addressing high per-device data volume, many device types, bursty arrivals, real‑time monitoring/alerts, and historical storage for reports and AI. Explains architectural choices and how ClickHouse and Apache Ignite are used in the solution — ClickHouse for analytical/historical workloads and Apache Ignite for in‑memory operational state and real‑time reaction.

AI-written summary · about

" chapters (14)

1. 0:00Introduction
2. 2:35Talk goals
3. 4:24Agricultural tech evolution
4. 10:52Agrotech challenges
5. 13:04Technical requirements
6. 16:15Classic architecture
7. 18:45Message broker introduction
8. 20:54PostgreSQL to ClickHouse
9. 26:10Lambda architecture need
10. 29:32Hot layer solution
11. 33:30Operational monitoring
12. 35:59Critical situation handling
13. 37:49Apache Ignite in system
14. 40:42Summary

" related talks

• 49mBeyond the Data Grid: Fast Data Processing with Apache IgniteDmitriy Setrakyan · GOTO Chicago 2015
• 33mDistributed Database DevOps Dilemmas? Kubernetes to the rescue!Akmal Chaudhri · ContainerDays Hamburg 2018
• 17mBuilding Real-Time Data Lake with ClickHouseAlexander Zaitsev · PG Armenia x Percona University – Yerevan, August 28, 2025
• 47mHunting issues in a Java app with 31G heap and 500G off-heap: an Apache Ignite case studyDmitriy Pavlov · JPoint 2024
• 27mSucceeding with Apache Druid and Clickstream DataPeter Marshall · OSA Con 2021
• 17mUsing ClickHouse® Database to Power an Analytics and Customer Engagement PlatformPrafulla Gupta · OSA Con 2022
• 07mIn-Hadoop Analytics: Bringing analytics to big dataAnjul Bhambhri · Strata Conference 2014
• 34mCatalogs, Iceberg, and Delta: Building the Foundation for Lakehouse AnalyticsZoe Steinkamp · Øredev 2025