The Real-Time Analytics Stack
summary
Argues a lean, cloud‑native RTA stack that replaces Lambda with a simple pipeline: events into Kafka, stream joins via Kafka Streams, materialized into Apache Pinot, and dashboards built with Streamlit. Demo generates ~51,000 orders, ~1,000 orders per minute and ~227,000,000 rupees revenue in the stream, shows real‑time joins (orders+products → enriched items) and low‑latency SQL queries updating every few seconds — the pipeline works end‑to‑end at interactive speeds.
AI-written summary · about
chapters (9)
- 0:00Start
- 0:35What is Real-Time Analytics?
- 10:43The Real-Time Analytics Stack
- 11:32Event Producers
- 15:28Streaming Platform: Technology choices
- 18:32Stream Processor: Technology choices
- 21:02Serving Layer: Desired Properties
- 23:05Serving Layer: Technology choices
- 25:11Front End: Technology choices
related talks
31mReal-time Analytics: Going Beyond Stream Processing With Apache Pinot
34mReal-Time Analytics: Going Beyond Stream Processing with Apache Pinot
27mStream, Store, Visualize: Fast-Track Real-Time Analytics the OSS Way
46mOptimizing Speed and Scale of User-Facing Analytics w Kafka and Pinot
47mReal-time analytics for your users with Apache Pinot
38mReal-Time Analytics: Going Beyond Stream Processing with Apache Pinot
49mIntroduction to Real-Time Analytics with Apache Pinot
47mBuilding a Real-Time Analytics Database