FlinkNDB : Skyrocketing Stateful Capabilities of Apache Flink
summary
FlinkNDB replaces embedded RocksDB with an external state backend built on MySQL Cluster Engine and demonstrates stateful scaling to hundreds of terabytes, rescaling and reconfiguration in sub-second time, and recovery without reloading node-local state. It reports <1ms reads, <0.01ms writes and 99.9999% availability, implements strict multi-version control for unbounded historical rollback, exposes state for external queries, and shows throughput and failure-recovery behavior using NEXMark benchmarks and targeted experiments.
AI-written summary · about
chapters (5)
- 0:00Introduction to FlinkNDB
- 7:00State management in Flink
- 14:07Liquid snapshots and checkpoints
- 20:16Data recovery and migration
- 23:36Q&A and conclusion
related talks
23mBetter, Faster, Stronger Streaming: Your First Dive into Flink SQL
36mApache Flink in the Cloud-native Era
35mApache Flink 2.0: Unified Enterprise Data Processing System and Beyond
59mMySQL (NDB) Cluster Best Practices
42mApache Flink deep-dive #bbuzz
39mA deep dive into Flink SQL
28mRundown of Flink's Checkpoints
37mBuilding and operating large-scale streaming applications at King