A brief history of time with Apache Flink
summary
Bouygues Telecom deploys a Flink streaming pipeline reading from Kafka, enriching and grouping events, and writing KPIs to HBase with historical Parquet snapshots on HDFS; production footprint reaches 10 billion events/day, ~400,000 eps peaks, ~300 users and 30 Flink jobs with sub-60s latency. Custom watermarks and triggers handle heavy out-of-order and late arrivals; alerts are emitted from Flink to legacy systems via SNMP. HBase causes analytical bottlenecks, so historic queries use Parquet and the roadmap moves toward Flink + Kafka + HBase/Kudu for faster analytics.
AI-written summary · about
chapters (21)
- 0:00Intro
- 0:29Agenda
- 0:51About Big Telecom
- 2:20Big Data Platform
- 3:18User Count
- 4:02Data Value
- 6:19Connect the dots
- 6:55Use case
- 9:25Use case summary
- 10:03Challenges
- 10:57Time management
- 12:42Future of Flink
- 13:48Flink Architecture
- 18:08Application Detail
- 19:10Production Metrics
- 19:31Global Network Operating Center
- 21:35Benefits
- 22:26Difficulties
- 23:28What next
- 24:19Alarms
- 25:08SNMP
related talks
34mKeynote | Flink Ahead 2.0: The Sequel
35mKeynote: Stream analytics made real with Pravega and Apache Flink
29mWhen your stream doesn't stream very well
27mArchitecting Sub-Second Analytics Pipelines
22mBatching Was Yesterday: Real-Time Tracking & Analysis For 100+ Million Visitors
20mBuilding realtime efficient queries in an IOT platform
36mBuilding a Self-Service Streaming Platform at Pinterest
19mLarge Scale Real Time Ad Invalid Traffic Detection with Flink