Keynote Panel: Future of Cluster Management
summary
Operators describe running Mesos at scale: Yelp runs it for all production and CI, Uber migrates most production in ~18 months with "big success," and Netflix open-sources its scheduler library Fenzo. Practical workarounds include static cluster partitioning, extensive out-of-band tooling for rolling upgrades and autoscaling, and experiments with cost-aware scaling on AWS spot pricing. Recurring problems are noisy neighbors, weak isolation, and capacity prediction; one successful prototype is automatic JVM tuning using blackbox learning.
AI-written summary · about
chapters (11)
- 0:00Introductions
- 3:00How important is open source
- 5:40Challenges
- 7:34Waiting
- 10:05Maintenance
- 13:43Buildin
- 14:42Mixed workloads
- 17:57Incremental improvements
- 19:37Improving isolation
- 20:27Intelligence
- 24:53Conclusion
related talks
23mHow cluster managers affect the landscape of modern distributed computing
43mSystems @Scale Summer Q&A Week 2
59mBuilding Distributed Systems with Apache Mesos
21mKeynote: CERN Experiences - Ricardo Rocha & Clenimar Filemon
19mKeynote: Making and Keeping Netflix Highly Available
01h 03mScaling Web Operations with Distributed Cluster Schedulers
19mKeynote Panel Discussion: SMACK in the Enterprise
33mDistributed Scheduler Hell