XSched: Preemptive Scheduling for Diverse XPUs
summary
XSched implements a preemptible command-queue called XQ and a multi-level hardware model (three levels: pending, inflight, running) to bring preemption to GPUs, NPUs, ASICs, and FPGAs; porting requires only a few hundred lines of C++ and XSched adapts to ten different XPUs across seven software stacks. Implementation tricks include binary rewriting for Nvidia, an Intel MPU hardware-assisted deactivation, and a modified GPU trap plus undocumented ioctl to trigger interrupts; results show up to 2× tail-latency reduction, 9× video-frame tail improvement, and a 1.5 overhead on average for bandwidth partitioning.
AI-written summary · about
chapters (12)
- 0:00XSched and XPUs intro
- 1:21Multitasking challenges on XPUs
- 2:23Native scheduler limits
- 4:11Move scheduling to host CPU
- 5:15General scheduler challenges
- 6:43XQ preemptable command queue
- 8:09Preemption hardware model
- 11:24XSched framework architecture
- 12:26XSched performance gains
- 13:29Video conference impact
- 13:50Triton integration comparison
- 14:13Conclusion and future
related talks
16mTowards Optimal Rack-scale μs-level CPU Scheduling through In-Network Workload...
35mSIG Scheduling Deep Dive
20mAdvancing HPC The Hybrid Scheduler Approach at the D.E.Shaw Group
25mApproximating Fair Queueing on Reconfigurable Switches
01h 07mThe Asynchronous C++ Parallel Programming Model
22mThe Token Slice: Implementing Preemptive Scheduling Via Chunked Decod...
11mXerxes: Extensive Exploration of Scalable Hardware Systems with CXL-Based Simulation...
14mPollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learning