Five years of highs and lows with Kafka
summary
An experience report from Mirakl about five years using Apache Kafka as the primary microservice messaging system, describing what worked and what failed. Covers concrete problems and anti-patterns: dead-letter handling, message delivery guarantees, message ordering, and a real incident where a single Kafka message caused all clients to crash. Includes practical do's and don'ts based on their production experience.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 1:06Why Kafka at Miracle
- 7:04First Kafka iteration
- 11:38Multi-tenant topics
- 17:33Dead letter and retries
- 24:29Lost messages and fix
- 30:05Internal Kafka library
- 32:14Producer antipatterns
- 34:40Consumer antipatterns
- 38:50Naming and partitions
- 42:19Critical Kafka settings
- 45:25Kafka as database
- 47:52Q&A
related talks
44mFive Years of the Good and the Not-So-Good with Kafka
39m7 Years in Kafka Land
57mWhen Everything Went Kafka
38mKafka Message Broker Under High Load
02h 48mKafka: Map of Pitfalls for Developers and Ops
53m(Spring) Kafka - one more arsenal in a distributed toolbox
39m10 years of Kafka. Tips, tricks and lessons learned
53mDemystifying Kafka