A Design for Massively Parallel Gibbs Sampling on the GPU via Static and Dynamic(…)
summary
Design enables massively parallel Gibbs sampling on GPUs by combining a factor-based intermediate representation for static graph-coloring to expose variable-level parallelism with reified schedule objects that encode hierarchical parallelism (steps, streams, parallel vs serial aggregation) and support autotuning. Implementation generates GPU code and, on benchmarks (Ising, HMMs, Bayesian mixtures), reports orders of magnitude improvements by switching between per-variable, intra-update, and hierarchical parallel strategies depending on workload characteristics.
AI-written summary · about
chapters (8)
- 0:00GPU-based Gibbs sampling intro
- 1:02Gibbs sampling fundamentals
- 3:06Markov blankets and parallelism
- 4:51Chromatic Gibbs sampling
- 5:55Workload differences in Gibbs
- 7:19Programmable Gibbs sampler design
- 8:23Schedule object structure
- 9:30Q&A on schedule optimization
related talks
30mScalable Graph-Parallel Inference Algorithms and Systems
01h 06mFoundation of GPU Programming
10mGraph Based Pipeline Parallelism
23mDynamic inference in probabilistic graphical models
17mServing Heterogeneous Machine Learning Models on Multi-GPU Servers...
15mMGG: Accelerating Graph Neural Networks with Fine-Grained Intra-Kernel Communication...
13mARK: GPU-driven Code Execution for Distributed Deep Learning
18mFast Inference for Probabilistic Graphical Models