Panel discussion: The new RabbitMQ Streams
summary
New RabbitMQ Streams implement an append-only log with non-destructive consuming so multiple readers can replay history and handle large fan‑outs. The broker writes segmented files (~0.5 GB) named by first offset, replicates via quorum commits, uses kernel sendfile for low overhead, and relies on OS fsync (quorum queues if you need fsync safety). Benchmarks show orders of magnitude throughput differences (classic ~40k/s versus ~1,000,000/s with stream protocol). Stream protocol listens on port 5552; Java and Go clients are available, .NET and Python are planned.
AI-written summary · about
chapters (22)
- 0:00Intro
- 1:40Gabriel
- 2:15What are streams in driving MQ
- 4:10Key new features
- 5:57Large fanouts
- 6:58What doors are being opened
- 9:50How do streams compare to classic use
- 12:57quorum queues and streams
- 15:40quran vs stream Q
- 16:45Why stream protocol
- 19:19Can stream protocol be used on embedded devices
- 21:45Roadmap for client libraries
- 23:40Queue features
- 26:28Streams protocol
- 27:56Socket reader
- 29:56Message storage
- 34:00Partition handling
- 35:38Data safety
- 37:15Plugins
- 38:58Metrics
- 41:37Stable release
- 43:47Recommendations
related talks
42mWhat's New in RabbitMQ Streams?
26mBreaking Magical Barriers
15mUpdate from RabbitMQ Core Team
37mLet's build our own message streaming platform
57mGame of Streams: How to Tame and Get the Most from Your Messaging Platforms
51mKeynote: Quorum Queues: A Retrospective
59mGame of Streams: How to Tame & Get the Most from your Messaging Platforms
28mConversational big data