Efficient Transparent Checkpointing of AI/ML Workloads in Kub...
summary
Extends Kubernetes checkpointing with CRIU plugins and a CUDA/AMD GPU checkpoint path to capture unified CPU+GPU state without API interception, producing ~11 GB archives that take about 1:04 to write and transfer, converts checkpoints into OCI images, and restores on another node to resume an LLM (demo with Llama 3.3 on an A10) cutting cold start from ~40s to ~25s. Works with containerd and cri-o, supports TCP close/reconnect options and integrates via a kubelet‑only API and Creo toolchain.
AI-written summary · about
chapters (8)
- 0:00Introduction and motivation
- 1:30Workload insights
- 5:39Proposed GPU checkpointing
- 7:24Technical implementation
- 11:19Hot-swapping demonstration
- 14:04Kubernetes migration demo
- 22:26Project roadmap
- 25:44Conclusion and Q&A
related talks
11mLightning Talk: Enabling Hot Restart of Stateful Applications Including GPU-Accelerate...
15mUniversal Checkpointing: A Flexible and Efficient Distributed Checkpointing...
13mByteCheckpoint: A Unified Checkpointing System for Large Foundation Model Development
22mBeyond Single-Cluster Limits: Scaling GPU Workloads Across Kubernetes With…
27mHarnessing Kubernetes for Scalable AI/ML Workloads
17mCheckmate: Zero Performance Overhead Model Checkpointing via Network Gradient Replication
16mCheck-N-Run: a Checkpointing System for Training Deep Learning Recommendation Models
31mAccelerate AI adoption: automated cloud efficiency