Sh... Fail Happens: Fail-aware Events Processing on AWS
summary
Covers concrete failure modes on AWS: SQS and Kinesis consumers, Lambda Event Source Mapping, and practical fixes like using dead-letter queues with 14-day SQS / 24h Kinesis retention and adding visibility-timeout based exponential backoff with jitter. Warns that reserved concurrency can drop messages and that event-source scaling is now 300 concurrent executions per minute. Recommends circuit breakers backed by ElastiCache/Momento/DynamoDB, enables the bisection boolean for batch splitting, uses AWS Fault Injection Simulator and SSM for tests, and shows Kinesis cost wins at ~40M/day.
AI-written summary · about
chapters (10)
- 0:00Intro
- 2:27Drop
- 10:56Retry
- 22:48Order
- 28:51Testing
- 31:10Observability
- 36:05Auditability
- 36:50Cost
- 38:36Key takeaways
- 39:21Outro
related talks
53mBottlenecks in Serverless Applications
39mBuilding Resilient Serverless Applications
51mServerless architectural patterns
40mThe Art of Embracing Failure at Scale
43mBest practices for continuous testing and observability
25mHow to Fail at Serverless (without even trying)
34mChaos engineering for serverless
44mBig Data with AWS: our lessons learned using managed services