Scrubs Scheduling in Reef and Beyond
summary
Quincy introduces a new scrub scheduler that lands in Reef: each PG now stores separate shallow/deep targets instead of flag blobs, the queue sorts by a single urgency field then target time, and a not before gate delays retries after failures. Chunk size defaults split shallow vs deep (larger shallow chunks), async retrying of failed reservations replaces the old penalty bench, and scrub resource budgets gain separate counters. Timely status updates, slow-replica warnings, and snap-detection checks are already implemented; defaults stay conservative.
AI-written summary · about
chapters (6)
- 0:00Start
- 8:31triggers & types
- 12:40sup. 1: chunk sizes
- 25:02a queue of scrub targets
- 37:40Reservation failure
- 43:43Scrub budget
related talks
39mIntro: Scheduling SIG
28mA Deep Dive into Ray’s Scheduling Policy
27mWhat does "high priority" mean? The secret to happy queues
28mFair by design: orchestrating background jobs in Ruby
35mLooking at SRE Needs and Trends over Two Decades with a...
40mEverything Will Flow
40mThe architecture evolution of Apache DolphinScheduler
33mMind Your P’s and Queues