Five Years of the Good and the Not-So-Good with Kafka
summary
Mirakl shares five years of operational Kafka experience: they initially used per-tenant topics (300+ topics) which caused scaling pain, then moved to functional topics holding multi-tenant data which avoids topic explosion but requires careful ordering and dead-letter handling. They describe a catastrophic incident where one malformed message and retry loops created a massive "dead-letter storm," and recommend concrete reliability patterns: transactional DB-first persistence of outgoing messages, a dead-letter manager with retry limits, tuning partitions and keys, and a shared library to enforce producer/consumer configs.
AI-written summary · about
chapters (11)
- 0:00Introduction to Kafka
- 1:48Initial Kafka issues
- 6:18First single-tenant architecture
- 10:09Functional multi-tenant architecture
- 17:35Handling failed messages
- 20:21Ensuring message deliverability
- 25:59Industrialization and internal library
- 28:45Common antipatterns
- 35:48Best practices before production
- 40:00Monitoring and maintenance
- 42:32Summary and conclusion
related talks
51mFive years of highs and lows with Kafka
53mDemystifying Kafka
45mAdvanced Kafka Features: From Dynamic Consumers to Fault Tolerance
39m7 Years in Kafka Land
38mKafka Message Broker Under High Load
05mIntroduction to Kafka Internals
53m(Spring) Kafka - one more arsenal in a distributed toolbox
48mKafka as a Message Queue: can you do it, and should you do it?