SEDA + Kafka in large-scale streams: Via Varejo's Black Friday case

on YouTube ↗45m

" summary

How Via Varejo handled over 50 million messages during Black Friday 2018 by combining the SEDA (Staged Event-Driven Architecture) pattern with Apache Kafka to build a resilient, highly available large-scale streaming pipeline. The talk describes implementation details and operational practices used—Dead Letters, monitoring, CorrelationID, logging—and scalability and fault-tolerance techniques for Kafka, plus lessons learned from running millions of messages per day.

AI-written summary · about

" chapters (9)

1. 0:00Introduction
2. 7:23Black Friday goals
3. 10:21Legacy system limits
4. 15:44Kafka and SEDA intro
5. 21:47Message aggregation
6. 27:42Consolidation and integration
7. 33:26Performance results
8. 38:12Key takeaways
9. 43:44Q&A

" related talks

• 53mBuilding Event Driven Services with Apache Kafka and Kafka StreamsBen Stopford · Devoxx Belgium 2017
• 27mKeynote II: Kafka and the Rise of Event-Driven MicroservicesNeha Narkehede · Scale By The Bay 2018
• 30mEvent Streaming and the Future of ApplicationsBen Stopford · JAX 2019
• 38mKafka Message Broker Under High LoadArtem Vybornov · РИТ++ 2017
• 53m(Spring) Kafka - one more arsenal in a distributed toolboxNakul Mishra · Spring I/O 2018
• 05mPREVIEW: 1 Day, 1 Data Hub, 100 Billion MessagesYuto Kawamura · Kafka Summit San Francisco 2017
• 18m"Kafka's Metamorphosis"Nick Grysimov · RubyConfBY 2019
• 46mLessons Learned From Working with 2000 Event-Driven MicroservicesNatan Silnitsky · Build Stuff 2022