PilotFish: Harvesting Free Cycles of Cloud Gaming with Deep Learning Training
summary
PilotFish co-locates deep learning training with cloud gaming, hooks DirectX/OpenGL to capture idle GPU periods in sub-millisecond granularity, and schedules training kernels only when they can finish before the next frame. It uses a low‑priority GPU stream, checkpointing, PCIe reservations and IO namespace isolation, and preempts straggler kernels with 0.7 ms overhead. Evaluation on popular games and models harvests up to 85.1% of idle GPU time while preserving game FPS and rendering quality.
AI-written summary · about
chapters (15)
- 0:00Intro
- 1:10Low GPU Util. of Cloud Gaming
- 2:23GPU Rendering 101
- 4:08Requirements for Co-location with Games
- 4:39DL Training is a Good Choice for Co-location
- 6:02Real-time capturing idle GPU periods
- 6:42Fine-grained Scheduling of DL operation
- 7:16DL Training Task Executor
- 8:24Avoding contention on other resources
- 10:31Evaluation (cont.)
- 11:18Evaluation Result
- 12:07Source of improvement dynamic scheduling
- 12:46Different harvest ratios for different models
- 13:59Soft/Hard Guanratee to Games
- 15:49Demo
related talks
24mHow to Eliminate the I-O Bottleneck & Continuously Feed the GPU While Training in the Cloud
26mDeepSpeed: Efficient Training Scalability for Deep Learning
38mMinimizing GPU Cost for Your Deep Learning on Kubernetes
17mYour Player is Smarter Than You Think: Self-Learning in Adaptive Streaming
36mAccelerating AI training fleets with sched_ext
16mTowards GPU Utilization Prediction for Cloud Deep Learning
38mMinimizing GPU Cost for Your Deep Learning on Kubernetes
17mPPipe: Efficient Video Analytics Serving on Heterogeneous GPU Clusters via...