PREVIEW: 1 Day, 1 Data Hub, 100 Billion Messages

on YouTube ↗05m

" summary

LINE feeds 100+ billion daily messages into Kafka for messaging, analytics and backups, with producers sending structured events, HTTP RPC payload logs and HBase write‑ahead mutations. They build Kafka Streams jobs that split huge master topics into subtopics to reduce consumer load, run Apache Storm for abuse detection, and ship data to Hadoop. Production broker issues get diagnosed with SystemTap, a fix surfaced as KAFKA-4614.

AI-written summary · about

" chapters (3)

1. 0:00Introduction
2. 0:50Outline
3. 1:47Kafka

" related talks

• 01h 00mPutting Apache Kafka to UseJay Kreps · Philly ETE 2015
• 38mKafka Message Broker Under High LoadArtem Vybornov · РИТ++ 2017
• 01h 31mDeveloping Real-Time Data Pipelines with Apache KafkaJoe Stein · SpringOne2GX 2015
• 39m400 Billion Messages a Day with KafkaMAULIN VASAVADA, Kevin Lu, Na Yang · Silicon Valley Code Camp 2018
• 41mApache Kafka simply explainedOlena Kutsenko · Berlin Buzzwords 2022
• 50mApplication development and data in the emerging world ...Neha Narkhede · Berlin Buzzwords 2016
• 50mIntroduction to Apache KafkaJames Ward · Devoxx US 2017
• 36mReal Time Big Data Analytics with Kafka, Storm, HBase #bbuzzAmeya Kanitkar · Berlin Buzzwords 2015