Apache Pulsar: Finally an Alternative to Kafka?
summary
Claims that Pulsar unifies messaging and streaming with separate broker and storage nodes are demonstrated with the partition‑free storage model and BookKeeper bookies, enabling independent scaling of compute and storage and immediate use of added disks. Built features include tiered storage to S3 and bi‑directional geo‑replication; these are production‑ready and battle‑tested (Yahoo origin, 10+ years). Community and adoption numbers: ~600 contributors, ~1,000 companies, ~10,000 Slack members. ZooKeeper is used today for coordination.
AI-written summary · about
chapters (20)
- 0:00Introduction
- 1:54Speaker background
- 3:19Streaming versus messaging
- 10:21Kafka as message queue
- 17:54Pulsar as unified platform
- 18:35Scalability in Pulsar
- 21:00Pulsar broker and bookie
- 22:46Partition-free storage
- 25:42Storage growth handling
- 29:38Replication and tiered storage
- 31:26Geographic replication
- 32:09Multi-tenancy
- 33:15Kafka compatibility
- 34:41Open source benefits
- 36:33Pulsar history
- 37:15Summary
- 37:59Q&A: Why Kafka
- 39:51Q&A: Pulsar downsides
- 41:36Q&A: Pulsar on AWS
- 42:58Q&A: Self-managing Pulsar
related talks
45mApache Pulsar: Finally an Alternative to Kafka?
50mApache Pulsar 101: architecture and features
32mStreaming Analytics with Apache Pulsar
10mA tour of Apache Pulsar
51mEvent Streaming and Processing with Apache Pulsar
22mBuilding FinTech Streaming Applications with Pulsar
27mLeveraging Event Streaming to Super-Charge your Business Solutions.
31mThe Power of Apache Pulsar. Harnessing Dapr to Build High Scale Messaging