Scalable by Design: How to Build an Elastic Cloud Infrastructure...
summary
Ad tech backend serves 3 billion requests daily and targets 500 million devices, running Go and some Rust, with 99% of infra in Terraform and heavy use of ECS/EKS, Kafka, Flink, Elasticsearch, Loki and S3 parquet lakes. Key results: spot instances cut infra cost by ~60%. Ongoing migrations include Redshift → Druid, SNS/SQS/Kinesis → Kafka, DynamoDB → Sila, and CloudWatch → Prometheus as part of a push to be cloud-agnostic by end of 2024.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 2:01Scalability challenges
- 5:38Infrastructure evolution learnings
- 8:50Design principles
- 11:41Current architecture
- 18:46Event bus, data lake & ML platform
- 20:58Spot instances & cost optimization
- 22:22Hyperscalers vs self-managed
- 25:16Cloud agnostic strategy
related talks
51mArchitecting for Hyperscale with AWS: Navigating Real-world Serverless Challenges
43mTools and Team for 20 Billion PageView/Month
48mCloud FinOps & Kubernetes Optimisation at Scale
28mCloud The Final Destination
46mArchitecting for Success and Runway
29mHow local businesses are rapidly achieving global scale on AWS
27mThe journey from legacy to cloud - native success
39mLarge scale streaming analytics