Connecting Reactive Applications with Fast Data Using Reactive Streams
summary
Demo shows a reactive app generating random points to estimate Pi and a Spark Streaming job processing 1s mini-batches; without flow control scheduling delay and memory backlog grow until results lag by tens of seconds. Spark estimates per-batch throughput and uses a PID-based rate estimator to throttle receivers, preventing backlog. Integration uses Reactive Streams interfaces to carry back-pressure signals and Spark Streaming back-pressure (since 1.5); options for overload are drop low-value events or scale the cluster.
AI-written summary · about
chapters (8)
- 0:00Fast Data and Reactive Apps
- 2:52Demo: reactive and fast data
- 9:47Spark Streaming micro-batches
- 13:24Back pressure in Spark
- 18:42PID for rate control in Spark
- 21:36Connecting with Reactive Streams
- 24:52Scaling streaming systems
- 26:43Summary and Q&A
related talks
30mRetrofit Your Java App With a Reactive Flow Pipeline
53mGetting into the Flow: building applications with Reactive Streams
47mBuilding High Performance Real-time Stream Processing Microservices
49mReactive Spring
20mBatching vs. Streaming - Scale & Process Millions of Measurements a Second
52mGoing Reactive with Helidon!
36mStream All The Things!
43mEvent Streams & Data Rapids: Harnessing the Full Power of Messaging Platforms.