Simulating Our Apache Mesos Framework for Fun and Profit and...
summary
Two Sigma builds a deterministic, faster-than-real-time simulator for Cook that uses an in-memory datastore and a mock Mesos master and driver to run historical job traces (May 1–6) and iterate quickly. Changing two knobs—the number of tasks to preempt per cycle and preemption selectivity—reduces waste by about 15% waste reduction while fairness drops only slightly; the same settings lower production waste after deployment. The simulator finds long-standing bugs, speeds experiments on millions of compute-hours, and integrates with CI.
AI-written summary · about
chapters (23)
- 0:00Agenda
- 0:41Example System
- 2:36Define: Experiment (noun)
- 3:41Being scientific
- 4:32What can we change?
- 6:24your system
- 9:29Experiment: Test upgrade
- 10:16Experiment: Double request rate
- 10:36Hypothesis: Server failure
- 11:03Examples
- 13:47Cook at Two Sigma
- 16:58Cook Architecture
- 18:10Mesos Architecture
- 19:06Building a Simulator: Choices
- 20:42Building a Simulator: Decisions
- 21:05Mock Mesos
- 21:47Trigger-able Cook Internals
- 22:31Simulation driver cycle
- 24:14How to handle time?
- 25:27Applied Simulation: Changing Pre-emption
- 26:09Preemption knobs
- 27:52Results in simulation: Fairness
- 28:40Results in production
related talks
41mEfficient cluster resource management by using Cook and Mesos
41mEfficient cluster resource management by using Cook and Mesos
59mBuilding Distributed Systems with Apache Mesos
58mDistributed Scheduling with Apache Mesos in the Cloud
38mMesomatic: The Cluster is a Library
46mWriting Stateful Frameworks
35mOpening the Box: Diagnosing Operating-System Task-Scheduler...
46mWhat do I do with 1000 cores