Optimizing Speed and Scale of Real-Time Analytics Using Apache Pulsar and Apache P...

on YouTube ↗33m

" summary

Uses Apache Pulsar as high-throughput ingestion and Apache Pinot as a real-time OLAP store, with Pulsar offloading storage to BookKeeper, offering tiered storage, geo-replication, multi-tenancy, functions and IO connectors. Proven at LinkedIn and Uber to serve 250k+ QPS while ingesting 1M+ events/sec, with user-facing queries targeting <100 ms latency and seconds freshness. DataStax runs a managed Pulsar (Astra Streaming) and StarTree brings Pinot’s index optimizations for sub-second interactive analytics.

AI-written summary · about

" chapters (5)

1. 0:00Start
2. 27:08Native Geo-Replication
3. 27:38Multi-Tenancy
4. 28:05Flexible Message Processing Model
5. 31:07Resources - Apache Pulsar

" related talks

• 48mEvent Streaming and Processing using Apache Pulsar and Apache Pinot.Mary Grygleski, Karin Wolok · Devoxx Ukraine 2022: Secure our Future
• 29mRealtime Analytics Unleashed: Unlocking the Potential of Data at ScaleBarkha Herman · Shift Miami 2023
• 46mOptimizing Speed and Scale of User-Facing Analytics w Kafka and PinotK Wolok, T Berglund · Voxxed Days Zürich 2022
• 49mIntroduction to Real-Time Analytics with Apache PinotTim Berglund · GOTO Chicago 2023
• 44mUSER-FACING, REALTIME ANALYTICS USING APACHE PINOTNeha Pawar · Crunch Conference 2021
• 48mApache Pulsar: Event Streaming in The Cloud Native WorldMary Grygleski · GOTO Chicago 2023
• 55mEvent Messaging and Streaming with Apache PulsarMary Grygleski · Great International Developer Summit 2023
• 14mFast Data on-Ramp with Apache Pulsar on K8Timothy J Spann · Kubernetes Batch + HPC Day Europe 2022