Kafka Is Not Always the Answer: Making Smart AsyncAPI Architecture Decisions
summary
Teams assume Kafka by default but a return-label automation proves that assumption wrong: a simple API-to-Kafka pipeline spawns four retry topics plus a dead-letter topic, increasing offsets, partitions and ops overhead, while swapping to SQS with one DLQ cuts operational cost, gives one-click redrives and still meets latency and reliability needs (latency rated 6, reliability 8). A CHAP decision process—Collaborative, Holistic, Adaptive, Practical—uses weighted NFR scores, a 250 msg/s assumption, and short POCs to pick the right async tool.
AI-written summary · about
chapters (12)
- 0:00Async API architecture choices
- 1:11Kafka as golden hammer
- 4:23Overusing Kafka problems
- 6:08Alternatives to Kafka examples
- 7:12CHAP decision-making framework
- 10:01Evaluating tools with business requirements
- 11:53Ecosystem fit and POC
- 14:21Retrospecting and tuning
- 16:47When Kafka is right
- 17:30Breaking golden hammer mindset
- 18:11Key takeaways
- 18:35Q&A
related talks
27mOptimize Your API with Event-Driven Microservice Architecture.
02h 48mKafka: Map of Pitfalls for Developers and Ops
42mIt's like 10,000 streams when what you need is a queue
33mKafka Microservices with Kafka plus Akka and Kafka Streams
39mReacting to an Event-Driven World
01h 02mKafka for .NET Developers
53mDemystifying Kafka
39m10 years of Kafka. Tips, tricks and lessons learned