Systems @Scale Summer Live Q&A Session #4
summary
Live Q&A covers actionable details: Terraform enforces state locking, treats each resource as a transaction (apply stops on a failing resource and resumes after fix), and needs external tooling for event-driven runs. The private-cloud control plane separates human vs automation config and exposes explicit controller inputs/outputs. Global reservations rebalance in about 1–3 hours, regional fluidity currently takes weeks. Solvers use mixed integer linear programming, converge to a good solution in ~5 minutes after warm start (hard cap 20 minutes). Cost controls: autoscaling, expiration tags, tagging hygiene.
AI-written summary · about
chapters (15)
- 0:00Welcome to Systems @Scale Q&A
- 1:49Terraform’s declarative approach
- 3:58ICS vs Terraform vs Kubernetes
- 7:11Global capacity at Meta
- 9:21Key enablers for capacity
- 11:26ICS controller dependencies
- 13:41Terraform state & transactions
- 16:54Tracing limitations
- 19:46Automate Terraform with events
- 20:50ICS automation pathways
- 23:20Cloud cost mitigation
- 25:47Shift services across regions
- 27:31Solver performance & extensibility
- 30:22Decouple infrastructure modules
- 31:47Closing remarks
related talks
35mSystems Scale Summer Q&A Week 1
43mSystems @Scale Summer Q&A Week 2
23mWeek 2 Q&A
33mLessons learned using Temporal to run Temporal Cloud
18mAvoiding and Breaking Out of Capacity Prison
29mBeyond the State Machine: Scaling Order Orchestration
40mSupercharging Cloud Orchestration with Temporal Open Source
01h 16mInterview and Q&A