Run GenAI Projects at Scale Securely: From the Operating System to the MLOps Platform
summary
GenAI production needs an end-to-end secure stack: Ubuntu with Ubuntu Pro LTS (5 years, extendable to 12) plus livepatches for kernel fixes, Charmed QFlow and MLflow for MLOps, and OpenSearch vector DB for RAG pipelines. Practical risks include evasion, data poisoning and confidentiality leaks; Google reports 64% of orgs planning adoption and chatbots saving ~2+ hours per person daily. Deployment options include private/public cloud or edge with confidential VMs on Intel/AMD/NVIDIA silicon (partners include Google Cloud and Microsoft Azure) to protect data in use.
AI-written summary · about
chapters (10)
- 0:00Introduction and Ubuntu overview
- 1:26Why GenAI is exploding now
- 3:35What is retrieval augmented generation
- 7:14GenAI security threats explained
- 10:46Securing RAG components
- 12:36Deploying RAG securely
- 14:46Operational security and monitoring
- 16:12Confidential AI and data in use
- 19:06Confidential VMs for GenAI
- 20:16Best practices for secure GenAI
related talks
23mThe State of GenAI & ML in the Cloud Native Ecosystem
50mMLOps for a hybrid AI platform compliant with European law
31mProductionizing ML with ML Ops and Cloud AI
43mSecurity of GenAI & LLMs: A New Era of Ethical Hacking
24mHow to create and scale AI Products, MLOPs and Generative AI
47mEmbedding AI: Don’t Burn Your Fingers in the Heat of the AI revolution
26mRunning Secure AI/ML Experiment in a Unified MLOPS-DevSecOps Pipeline
42mMLOps Under Attack: Threat Modeling Modern AI Systems