7 Years in Kafka Land
summary
A seven-year Michelin experience report about adopting event-driven architectures using Apache Kafka as the event bus. It covers installing and operating initial Kafka clusters, running distributed architectures, migrating to a managed cloud service, designing event-based functional architectures, and using libraries like Kafka Streams. The talk also shares pitfalls, Kafka Streams best practices and open-source contributions, cloud migration strategies, and approaches to real-time streaming analytics and democratizing streaming app development.
AI-written summary · about
chapters (19)
- 0:00Introduction
- 1:262018 data ambitions
- 2:47Kafka production architecture
- 4:34First production incident
- 6:19Managed Kafka migration
- 7:21Migration strategy
- 9:302021 progress
- 10:12Event Driven Architecture
- 11:17Event modeling
- 13:01Kafka Streams challenges
- 16:51Streaming join complexity
- 17:34Castlify framework
- 19:58Current Kafka clusters
- 21:43Kafka client integration
- 22:47Operational vs analytical
- 24:10Streaming democratization
- 25:55Kafka and generative AI
- 27:39Conclusion
- 30:31Q&A
related talks
51mFive years of highs and lows with Kafka
57mWhen Everything Went Kafka
41mKafka as a Platform: The Ecosystem from the Ground Up
39mKafka as a Platform: The Ecosystem from the Ground Up
44mFive Years of the Good and the Not-So-Good with Kafka
41mKafka as a Platform: the Ecosystem from the Ground Up
54mKafka as a Platform: the Ecosystem from the Ground Up
53m(Spring) Kafka - one more arsenal in a distributed toolbox