Getting Real with NodeJS and Kafka: An Event Stream Tale

on YouTube ↗08m

" summary

Concrete config and code using KafkaJS with NodeJS: shows how to get exactly-once semantics inside Kafka (not in external databases) by setting producer acks=all, transactional.id, sending offsets in the transaction, disabling auto commit on consumers and limiting max in-flight requests to one to preserve ordering. Covers using a dead-letter queue and manual offset commits to avoid poison messages and explains that the cluster needs at least 3 partitions and 2 in-sync replicas; includes a docker-compose multi-node setup to reproduce and test the behavior.

AI-written summary · about

" related talks

• 01h 02mKafka for .NET DevelopersIan Cooper · NDC Sydney 2024
• 42mCreating event-driven microservices: the why, how and whatKatherine Stanley · Voxxed Days Microservices 2019
• 16mAn introduction to Apache Kafka - this is Event Streaming, not just MessagingKatherine Stanley · Voxxed Days Microservices 2019
• 01h 00mReacting to an Event-Driven WorldGrace Jansen, Katherine Stanley · Developer Summit 2020
• 16mIntroducing Apache KafkaKatherine Stanley · Devoxx Belgium 2019
• 01h 00mKafka for .NET DevelopersIan Cooper · NDC Oslo 2024
• 39mReacting to an Event-Driven WorldGrace Jansen, Katherine Stanley · jLove 2020
• 42mIt's like 10,000 streams when what you need is a queueChris Simon · NDC Copenhagen 2026