A Year of Diagnosing Pediatric Cancer with Event-driven Architecture
summary
Serverless AWS pipeline processes tumor normal whole‑exome plus methylation and Archer fusion for over ~2,000 pediatric cases in 18 months, using S3 event notifications, EventBridge, SQS/SNS, Step Functions, Lambda, Batch and EMR (moving to EMR Serverless), DynamoDB, Athena and Cognito. Built a decoupled tracking layer that matches output files to orders and ships deidentified data to NCI in minutes, replaced brittle Excel review with VHouse2 interactive variant viewer and a ScanIt annotation sync that propagates database updates to cases, enabling parallel validation runs and faster human curation.
AI-written summary · about
chapters (10)
- 0:00Introduction and context
- 1:45Genomics 101 for developers
- 6:44Molecular characterization
- 10:36Event-driven cloud pipeline
- 21:27Data visualization
- 22:53Data tracking and storage
- 25:55Human interpretation and security
- 31:51Lessons and challenges
- 33:52System modernization
- 40:35Conclusion and architecture
related talks
07mCloud architecture for population-scale analyses of genomes and AI.
32mGoing Server less for Web Services
27mVisualizing Cloud Systems
39mBuilding Highly Scalable Microservices with Serverless
42mEnterprise Serverless Adoption. An Experience Report
49mBuilding Next-Generation Applications with EDA
30mEvolving Serverless Architectures
25mData Gone in 60 Seconds: A Serverless ETL Heist