From hot data sources to interactive clients in a hurry
summary
Live code builds a voting backend that accepts rapid votes, aggregates them into 100 ms time windows with a scan-based reducer, and streams live results to browsers via Server-Sent Events; the pipeline uses a serialized PublishedProcessor, applies onBackpressureDrop to avoid OOM, and publishes to many subscribers. Tests inject a TestScheduler so windowing is deterministic. Front end uses D3.js for visualization and optional persistence to Apache Kafka; demo survives hundreds of thousands (attempted up to a million) synthetic votes but shows buffering limits on some setups.
AI-written summary · about
chapters (10)
- 0:00Interactive data sources
- 2:56Eurovision voting backend
- 9:02Vote controller endpoint
- 12:06Processing votes reactively
- 16:30Streaming results with SSE
- 20:23Slow clients and backpressure
- 21:49Real-time vote aggregation
- 24:22Testing reactive pipelines
- 27:01Live voting visualization
- 30:10Scaling with Kafka
related talks
38mBeyond REST: Using Full-Stack Signals for Real-Time Reactive UIs
28mBringing Data to your Client-Side Apps
21mEvent is the new Model
45mBeyond REST: Using Full-Stack Signals for Real-Time Reactive UIs
11mBuilding Real-Time Feature Pipelines
23mBeyond Virtual Lists: How to Render 100K Items with 100s of Updates/sec in React
09mMake your Next.js app realtime and collaborative
46mIntegration of Distributed Real-time Data Computing in a Web App