Deploying Batch and Streaming Architectures on AWS
summary
Shows how Amazon EMR, Amazon Kinesis, Amazon Redshift and AWS Lambda combine into a Lambda architecture with concrete numbers: Kinesis shard = 1,000 records/sec or 1 MB/sec, retention 24 hours; EMR task nodes on spot instances cut runtime and cost by adding transient workers; Redshift scales from 160 GB nodes up to 2 PB and uses columnar MPP storage for fast analytic queries; includes the spot–task pattern, bootstrapping EMR, and end‑to‑end data flow that actually runs.
AI-written summary · about
chapters (7)
- 0:00Start
- 1:44Agenda
- 6:24Lambda, Lambda, Lambda
- 8:35Lambda Architecture
- 15:15Cost & Time
- 17:00EMR with spot instances
- 24:18Getting data out of Kinesis
related talks
20mBatching vs. Streaming - Scale & Process Millions of Measurements a Second
44mBig Data with AWS: our lessons learned using managed services
37mBuild a Modern Data Platform on AWS
47mEvent-driven Serverless Architecture with DynamoDB
51mServerless architectural patterns
28mInnovations in AWS analytics: Data processing
43mFuture-proofing your applications with AWS databases
50mDeveloping Highly Scalable Image Storage Solution with AWS Serverless