The State of Flink and how to adopt Stream Processing
summary
Apache Flink powers large production pipelines and attendees report 2.1 trillion events/day processed by their Flink deployments; collaboration with Alibaba yields incremental checkpoints and Blink features, Netflix and IMG show massive real-time use cases including ML models and fraud detection. The push is toward stateful stream processing where state and computation cohabit, and a new Kubernetes-based application manager offers versioned applications, safe points, stateful upgrades, rewind/fork from snapshots and a demoed payments dashboard showing live repair via snapshot-based forks.
AI-written summary · about
chapters (7)
- 0:00Welcome and introduction
- 6:39Flink in 2017 overview
- 13:25Stream processing evolution
- 20:11Modern app architecture
- 26:11Stateful stream processing
- 33:56Application management demo
- 44:43Closing remarks
related talks
40mTowards Flink 2.0: Unified Batch & Stream Processing
20mFlinking, Fast and Slow
27mBatch/Stream Ask-Me-Anything
30mBlink: Improvements to Flink & its application in Alibaba Search
35mBeyond the Watermark: On-Demand Backfilling in Flink
01h 09mExploring Flow Architectures
36mMicroStream Labs – New Incredible Feature Preview
41mConcurrent Stream Processing