Keynote: Quorum Queues: A Retrospective

on YouTube ↗51m

" summary

Quorum Queues use Raft to put full queue state into a replicated log (released as a feature in RabbitMQ 3.8, with 3.10 adding message TTL and at‑least‑once dead‑lettering). Classic mirrored queues once lost 35% of confirmed messages in Jepsen tests, so this design trades consensus cost for safety, adds a centralized writer‑head that batches fsyncs, removes in‑memory message copies in 3.10, introduces machine versions for safe upgrades, enforces delivery limits to avoid unbounded log growth, and scales to thousands of queues with single‑queue throughput limits.

AI-written summary · about

" chapters (10)

1. 0:0006:00 - Quorum Queues timeline
2. 6:0113:19 - 2016 | What we needed?
3. 13:2016:40 - Consesus is Costly
4. 16:4118:52 - Availability
5. 18:5221:18 - Throughput
6. 21:1924:11 - Throughput vs Many Queues
7. 24:1225:42 - Throughput | Resource use: Local Delivery
8. 25:4329:40 - Maintainable
9. 29:4140:28 - Features
10. 40:2951:14 - QnA

" related talks

• 55mImplementing Raft in RabbitMQKarl Nilsson, Michael Klishin · SpringOne Platform 2017
• 37mImplementing Raft in RabbitMQDaniil Fedotov · Code Mesh LDN 2018
• 43mRa: a Raft implementationKarl Nilsson · Code BEAM Lite Berlin 2018
• 26mBreaking Magical BarriersGerhard Lazu · Systems @Scale Spring 2021
• 29mKeynote: The Power of QueuesDavid Ware · MQ Summit 2025
• 50mPanel discussion: The new RabbitMQ StreamsKarl Nilsson, Arnaud Cogoluègnes, Diana Parra Corbacho, Gabor Olah, Ayanda Dude · RabbitMQ Summit 2021
• 16mTo queue or not to queue, that is the question.Lovisa Johansson · Voxxed Days Brussels 2023
• 47mDistributed Systems: When Democracy and Computer Science Come TogetherIdan Shalom · QCon São Paulo 2018