Extending Mesos for Large Scale Clusters
summary
Uber extends Mesos in two concrete ways: it implements an operator API to push reactive updates into the service discovery stack, and it adds custom Mesos modules that enable controlled oversubscription of compute resources across many frameworks in a highly utilized shared cluster. The system integrates with existing schedulers, exposes HTTP hooks for immediate events, and ships monitoring, build/packaging and integration tests; Uber reports the changes run in production at cluster scale and improve utilization while preserving task isolation and restart semantics.
AI-written summary · about
chapters (8)
- 0:00Mesos cluster extension
- 3:30Scalable resource management
- 8:10Custom executors
- 13:46Task failure recovery
- 19:04Event stream processing
- 25:50API automation
- 30:52Isolation and tuning
- 31:40Q&A
related talks
30mDay 2 Operations Best Practices with Apache Mesos
42mMetrics at scale
45mMetrics at scale
40mMeasuring and Optimizing Kubeflow Clusters at Lyft
11mWelcome & Opening Remarks
28mOn-Call Experience at Uber
46mDeploying and maintaining software for today's infrastructure
42mMaking Resilient, Self-Adaptive, And Autonomous Kubernetes Clusters