Dataflow Engines for Executing Cloud Applications: a Maslow's Hammer or Natural Outcome?
summary
Argues that large-scale stateful serverless workloads map to dataflow graphs and run on existing engines, shows concrete examples (shopping cart, payments) implemented as partitioned operators with local keyed state to cut latency, uses Apache Flink and Beam-style semantics, demonstrates replay/time-travel debugging for event replay and debugging, and highlights missing pieces (messaging integration, migration, fine-grained service levels). Concludes dataflow is a practical path but current engines need targeted features before being a universal cloud execution layer.
AI-written summary · about
chapters (4)
- 0:00How do people build scalable cloud applications today?
- 9:15What are the main pain-points?
- 12:44Is it a good idea to leverage dataflow processors for cloud apps?
- 14:18Are current dataflow processors up to the task?
related talks
01h 00mDataflow Engines for Executing Cloud Applications: a Maslow's Hammer?
51mComplex Event Flows in Distributed Systems
38mDeploying Stateful FaaS on Streaming Dataflows
34mHands-on Stateful Serverless Applications with Kubernetes and Stateful Functions
35mHands-On Stateful Serverless Applications with K8s and Stateful Functions
50mChoreography vs Orchestration in serverless microservices
49mBuilding Next-Generation Applications with EDA
34mDesigning Data Intensive Applications In Serverless Architecture