Why it is harder to scale with stateful, data-driven systems
summary
Claim: state never disappears it moves and scaling stateful services is harder. Flipkart Flux is an open source stateful serverless orchestrator that runs up to one million workflows per day, provides at-least-once delivery with idempotent tasks, retries, sidelining and an operator console, and uses Akka cluster and sharded MySQL plus JMX/Grafana metrics and Hystrix timeouts. Design tradeoffs include local vs global retry coordination, cold-start limits, and a clusterwide scheduler singleton, so it works at scale but hits consensus and placement limits.
AI-written summary · about
chapters (13)
- 0:00Start
- 1:24Things you hear
- 4:09Truths
- 6:25State in e-Commerce systems
- 10:20Event driven Finite State Machine (FSM)
- 10:54Stateful EDA = Workflow
- 20:33Stateful Serverless platform
- 22:00Flux Programming Primitives
- 22:51Deployment Unit
- 23:19Fault tolerance
- 26:48Flux Design
- 27:43Tech choices
- 29:22Flux Software components
related talks
33mFull Stack Flux (was" The future of JavaScript layout systems")
27mEmpowering Resilient Scalability: The Power of Choreography in Distributed Systems
05mFluxCapacitor: A Highly Scalable App
36mManaging Thousands of Clusters and Their Workloads with Flux
33mHigh availability by offloading work
47mComplex event flows in distributed systems
43mIntro to Stateful Services
37mBeyond Microservices: Streams, State & Scalability