From Tremors to Saving Lives: Serverless Kafka Arch for Earthquake Notifications
summary
A serverless pipeline that combines Lambda Kafka producers/consumers with Confluent Kafka, SQS dead‑letter queues and Terraform/GitHub Actions autoscaling pushes SMS and email alerts within the 1–2 minute early‑warning window; it uses multithreaded producers, a topic roulette to spread load across partitions, and a custom DynamoDB timestamp locks + optimistic versioning to prevent double sends. Segmented subscriber processing and provisioned concurrency aim for parallel delivery; system is proven at ~500 test subscribers and targets 5,000–10,000 but not yet tested at city scale.
AI-written summary · about
chapters (13)
- 0:00Introduction and speaker background
- 2:38Need for earthquake notifications
- 5:08Finding real-time earthquake data
- 8:19Why choose serverless ecosystem
- 10:05First architecture overview
- 16:30Government data integration
- 20:08Lambda Kafka producer details
- 22:16Lambda Kafka consumer mechanisms
- 25:52Subscriber processing improvements
- 30:30Autoscaling and CI/CD pipeline
- 33:29Infrastructure as code approach
- 35:37Future plans and research
- 36:41Q&A session
related talks
36mLeveraging Serverless architecture with Confluent Kafka
28mBuilding Event Streaming Applications with Serverless Kafka
12mServerless Stream Processing with Confluent Cloud, Kafka and AWS Lambda
37mEvent-based, serverless app to broadcast real-time traffic alerts
47mServerless Kafka Patterns.
42mBuilding real-time serverless data applications
51mServerless architectural patterns
56mScaling Telemetry Systems with Streaming & Serverless