WIP title: Building Practical Privacy-Preserving Inference Systems
summary
Bolt implements privacy-preserving two‑party inference for BERT using MPC plus HE and cuts the extreme communication of prior work (Iron used 280 GB and 3.5 hours for BERT‑Base) by packing matrices diagonally and applying baby‑step giant‑step to reduce rotations, trading only ~5% extra compute for far fewer ciphertexts. Rotom is a tensor compiler that autovectorizes HE from a PyTorch frontend, generates HE kernels with layout alignment, compiles most benchmarks in minutes, and produces kernels that are up to 80x faster on BERT attention and match or improve Bolt’s hand‑tuned packing.
AI-written summary · about
chapters (12)
- 0:00Privacy-preserving inference intro
- 2:28Challenges in privacy systems
- 4:55BOLT for transformer inference
- 11:39Homomorphic encryption packing
- 18:45Limitations of manual protocols
- 20:31Rotom automated HE compilation
- 23:25Rotom workflow and layout IR
- 32:44Layout assignment and HE kernel
- 44:07Layout conversion operators
- 51:18Rotom evaluation vs prior systems
- 53:05Future directions in systems
- 54:12Q&A
related talks
15mBreaking the Layer Barrier: Remodeling Private Transformer Inference with...
13mLightning Talk: Exploring PiPPY, Tensor Parallel and Torchserve for Large...
26mEfficiently serving LLMs at scale.
11mCrypTen
40mPanel discussion "NLP"
26mThunder: Distribute and Optimize Your PyTorch Models With Zero...
16mPrivacy Preserving AI
31mShort Industry Talks pt. 1