Pinot, Why Are You So Fast?
summary
Apache Pinot claims ingest >1M events/sec directly from Kafka and serves interactive UI queries in single to tens of milliseconds by coupling columnar segments (~200–400 MB) with server-local compute, broker query routing, and controller metadata in ZooKeeper. Servers act as Kafka consumers with partition-aware segment placement, minions build segments from batch jobs, and the star-tree index stores precomputed aggregates to avoid scans. StarTree Cloud adds tiered S3 storage at modest latency cost.
AI-written summary · about
chapters (6)
- 0:00Introduction to Pinot
- 8:48Evolution of analytics databases
- 16:17How Pinot achieves speed
- 23:18Streaming data ingestion
- 27:11Pinot in modern architecture
- 30:02Q&A session
related talks
54mAn Introduction to Apache Pinot
46mOptimizing Speed and Scale of User-Facing Analytics w Kafka and Pinot
49mIntroduction to Real-Time Analytics with Apache Pinot
44mUSER-FACING, REALTIME ANALYTICS USING APACHE PINOT
52mMaking Kafka Queryable with Apache Pinot
39mFaster Analytics for Fast Data with Apache Pinot and Flink SQL
34mReal-Time Analytics: Going Beyond Stream Processing with Apache Pinot
47mReal-time analytics for your users with Apache Pinot