Spring Kafka beyond the basics - Kafka journey at ING Bank
summary
ING runs Kafka in production for over seven years with more than 5,000 topics serving over 1,000 development teams and roughly 1M messages/sec, growing ~10% monthly. Shows local single-node dev clusters, Confluent Schema Registry, Spring Kafka, and fixes for the classic poison pill (console-producer bypass + deserialization failures) using Spring Kafka’s error-handling deserializer and dead-letter topics. Demonstrates Prometheus/micrometer metrics, Zipkin distributed tracing, Kafka Streams topology test driver, and recommends unit-first testing over slow embedded or Testcontainers end-to-end tests.
AI-written summary · about
chapters (17)
- 0:00Introduction and team background
- 1:54Kafka at ING Bank overview
- 3:19Kafka nutshell explanation
- 7:31Spring Kafka introduction
- 12:08Poison pill scenario explained
- 16:47Fixing poison pill with Spring Kafka
- 22:05Dead letter topic configuration
- 26:19Runtime exception handling
- 29:29Demo of exception handling
- 31:40Testing Kafka applications
- 33:54Kafka Streams topology testing
- 36:21Integration testing alternatives
- 37:24Monitoring Spring Kafka apps
- 38:50Prometheus metrics demo
- 41:24Distributed tracing with Zipkin
- 43:13Lessons learned summary
- 45:41Q&A session
related talks
44mSpring Kafka beyond the basics - Lessons learned on our Kafka journey at ING Bank
38mWalking Through the Spring Stack for Apache Kafka
21mHow Apache Kafka® Generates Value In Retail With Confluent
16mIntroducing Apache Kafka
46mSpring Kafka. How to Prepare and Not Shoot Yourself in the Foot
01h 02mKafka for .NET Developers
54mMicroservices with Kafka
59mBuilding Data Intensive Microservices with Apache Kafka