Managing Nation-Wide Traffic Cameras and Sensors
summary
Belgium builds a nation-wide system aimed at 20,000 cameras (project scales up from 60) that sends plate, GPS and timestamp messages which cameras cache up to 7 days when offline. The pipeline uses Apache Kafka with Zookeeper for leader election and a Scala event generator built on Akka cluster and Akka Streams; a master–worker negotiation assigns continuous tasks, overtaking detection uses a separate topic keyed by segment, and once inside the system urgent events generate in ~50–100 ms.
AI-written summary · about
chapters (13)
- 0:00Introduction and background
- 1:54Camera data and constraints
- 4:46Original system limitations
- 5:51Lambda architecture design
- 7:54Event generator overview
- 8:57Event generator implementation
- 11:30Master-worker design pattern
- 12:53Zookeeper for leader election
- 15:46Worker implementation details
- 17:11Kafka partitioning strategy
- 18:36Overtaking event handling
- 20:03Testing and development setup
- 20:48Q&A and summary
related talks
49mScaling live transactional data at the Tour de France
43mBuilding a community fountain around your data stream
32mDetecting Events on the Web with Java, Kafka and ZooKeeper
30mBuilding a Large-Scale, Streaming-Based Logging and Monitoring Solution
45m1984 Big Brother is Watching you (and helping with savings)
35mA Service with Cameras - a Performance Minefield
11mCognitive and Self-Adaptive System for Effective Distributed-Tracing...
24mAutomated Root Cause Analysis