A tour of Apache Pulsar

on YouTube ↗10m

" summary

Apache Pulsar is a distributed pub/sub system develop at Yahoo! Architecture splits state in Apache Zookeeper, processing on Pulsar nodes and long‑term logs in BookKeeper; features include Pulsar Functions (lambda‑style on-node Java), a schema registry that validates and preserves bad records, tiered storage to S3/Glacier, and Pulsar SQL wired to BookKeeper for live SQL over streams. A connected‑home deployment proves it works but also surfaces brittle rules that trigger false alarms.

AI-written summary · about

" chapters (10)

1. 0:00Intro
2. 1:00Messaging
3. 1:42Other messaging systems
4. 1:58Longterm storage
5. 2:54History
6. 3:16Architecture
7. 4:01Pulsar Functions
8. 5:07Pulsar Schema Registry
9. 6:02Pulsar tiered storage
10. 7:20Pulsar sequel

" related talks

• 46mApache Pulsar: Finally an Alternative to Kafka?Julien Jakubowski · DevDays Europe 2024
• 50mApache Pulsar 101: architecture and featuresSteven Le Roux, Alexandre Duval · Devoxx France 2021
• 51mEvent Streaming and Processing with Apache PulsarMary Grygleski · Conf42 Cloud Native 2022
• 45mApache Pulsar: Finally an Alternative to Kafka?Julien Jakubowski · Big Data & RPA Conference Europe 2024
• 32mStreaming Analytics with Apache PulsarAddison Higham · Aerospike Digital Summit 2021
• 54mApache Pulsar Development 101Timothy J Spann · Devnexus 2023
• 22mBuilding FinTech Streaming Applications with PulsarTimothy J Spann · Open Source in Finance Forum New York 2022
• 31mThe Power of Apache Pulsar. Harnessing Dapr to Build High Scale MessagingHugo Smitter · AppDeveloperCon NA 2024