Introduction to Flink in 30 minutes

on YouTube ↗27m

" summary

A working mobile data pipeline is built and demoed: Kafka sources feed Flink in both batch and streaming modes, Flink SQL performs a temporal join (system_time as of) to enrich usage with per-account quotas, then a DataStream process function sums bytes per billing cycle and emits alerts at 90% of quota. State stays local and is snapshotted for recovery (exactly-once state snapshots). The demo processes hundreds of megabytes in about 30 minutes and the code is at https://github.com/alpinegizmo/flink-mobile-data-usage.

AI-written summary · about

" chapters (13)

1. 0:00Introduction and use case
2. 1:20Project requirements
3. 2:32Apache Flink overview
4. 3:48Flink architecture and APIs
5. 5:37Installation and word count
6. 7:42Job manager and task managers
7. 9:11Project bootstrapping
8. 10:17Batch and streaming jobs
9. 13:22Data enrichment with SQL
10. 17:01Implementing alerting logic
11. 20:38Monitoring with Flink UI
12. 22:05SQL client for analytics
13. 24:22Conclusion and resources

" related talks

• 46mFlink SQL in 2020: Time to show off!Timo Walther, Fabian Hueske · Flink Forward Virtual 2020
• 38mApache Flink: A Very Quick GuideVladimir Bacvanski · Data By the Bay 2016
• 23mBetter, Faster, Stronger Streaming: Your First Dive into Flink SQLCaito Scherr · DataEngBytes 2021
• 39mBetter, Faster, Stronger Streaming: Your First Dive into Flink SQLCaito Scherr · Data Con LA 2021
• 43mA Data Streaming Architecture with Apache Flink #bbuzzRobert Metzger · Berlin Buzzwords 2016
• 43mLeverage the power and simplicity of SQL on Apache FlinkFabian Hueske · Berlin Buzzwords 2018
• 49mscale.bythebay.io: Stephan Ewen, Apache Flink and the Next Wave of Stream Processing ApplicationsStephan Ewen · Scale By the Bay 2017
• 41mBeyond the Hype: When Apache Flink Solves Real ProblemsNaci Simsek · Berlin Buzzwords 2026