Towards Confidential AI for the Masses!
summary
Shows a working end-to-end confidential inference service that boots a CVM, attests CPU and GPU, mounts an encrypted volume, downloads an encrypted model from Hugging Face and serves it with an InstructLab-based server on Nvidia H100 and an AMD-based TEE. Uses kata/confidential containers, GPU-CC and host-side bounce buffers, which increase model load times and complicate multi-model serving; demo succeeds but highlights NVLink encryption gaps and the promise of trusted IO to remove bounce-buffer overhead.
AI-written summary · about
chapters (16)
- 0:00Confidential AI intro
- 1:05Why confidential computing matters
- 3:56Data in use and TEE
- 5:41Confidential computing for GPUs
- 9:52Deploying confidential AI
- 14:23GPU interconnect challenges
- 16:13Secure boot and attestation
- 19:42Distributed trust challenges
- 21:46Bespoke confidential AI solutions
- 23:30Kubernetes for confidential containers
- 26:36Live demo: confidential AI workflow
- 31:14Limitations of confidential computing
- 33:23Verification at scale
- 35:08Tooling for deployment verification
- 36:09Summary and future
- 37:13Q&A on trust models
related talks
24mStrictly Confidential: Use Cases & Demos of AI Workflows in TEEs
20mBuilding Trusted AI Factories with NVIDIA Accelerated Confidential Computing
36mCTO Panel: The Impact of Confidential Computing with Intel, Azure, AMD & NVIDIA
28mPreserving Data and AI/ML Model Privacy Using Confidential..
16mIntroduction to confidential computing
20mTrusted Hardware for AI Workloads: Extending Confidential...
15mKeynote: Trustworthy AI with Confidential Computing
44mGPU-less, Trust-less, Limit-less: Reimagining the Confidential AI Cloud