Apache Flink Worst Practices
summary
Practical checklist: choose DataStream for stateful apps, avoid Table API if you need stateful upgrades or late data handling, and pick delivery semantics deliberately — checkpointing exactly-once with replayable sources for correctness or at-least-once without checkpointing if some loss is acceptable. Measure feasibility by serialized state size and network (checkpoint bandwidth ≈ total serialized state size / checkpoint interval). Keep keys tiny, project early (store full payload as bytes only when needed), avoid custom threads/static shared state, use the State Processor API and MiniCluster/test harness for safe upgrades and tests.
AI-written summary · about
chapters (18)
- 0:00Intro
- 0:59Before you start...
- 2:10Training & Community
- 4:44Consistency & Delivery Guarantees
- 6:52Application Evolution
- 9:45Feasibility Check
- 13:16Business Requirements
- 18:01SOL or DataStream API
- 20:12Data Types
- 22:12Serialization ctd.
- 23:27Concurrency
- 26:23Windowing
- 28:12Queryable State
- 30:34Data Stream API Classics
- 31:41Testing
- 32:43Flink's Test Harnesses
- 33:44Go Live
- 38:07Maintenance
related talks
42mApache Flink Worst Practices
29mManaging State in Apache Flink
42mApache Flink deep-dive #bbuzz
36mApache Flink in the Cloud-native Era
26mHow Streaming Helps Your Staging Environment and Sandboxes Stay Up To Date
38mStreaming Fast and Slow
26m“Hit me, baby, just one time” – Building End-to-End Exactly-Once Apps w/ Flink
31mTale of Stateful Stream to Stream Processing