Multi-Layered Guardrails for Cloud Native AI: Enforcin...
summary
Claims a practical cloud-native pattern: deploy specialized agents as Kubernetes pods orchestrated with a service mesh, use LlamaIndex for the REACT-style pipeline, and enforce multi-layered guardrails at input, inference, and output without touching AI code. The demo uses SQuAD, DPL/DeepEval for automated scoring, and Envoy Lua filters at the network edge to block PII. Results show a transparency tradeoff — distributed gets transparency 0.55 but costs ~100s per query versus 22s for centralized; hybrid often runs fastest.
AI-written summary · about
chapters (10)
- 0:00AI agent guardrails challenge
- 3:57Enterprise AI agent trends
- 7:32Cloud native agentic systems
- 10:23Governance architecture principles
- 13:31Experiment setup with SQuAD
- 17:42Multi-agent system implementation
- 23:21Governance model performance
- 26:30Safety versus performance trade-offs
- 27:54Dynamic guardrails with Envoy
- 32:55Conclusion and CNCF call
related talks
26mSponsored Session: Navigating Open Source AI with Guardrails
25mSecuring AI Pipelines: Real-World Attacks on Kubernetes-Based AI Infrastructure
30mSecuring AI Workloads: Building Zero-Trust Architecture for LLM Applications
47mEmbedding AI: Don’t Burn Your Fingers in the Heat of the AI revolution
25mWhat We Learned Securing AI Agents at Scale on Multi-Tenant Kubernetes Clusters
38mSecurity for AI Agents Using an Ensemble of Fine-tuned... (Lidan Hazout, Bar Kaduri)
42mLLMs on Autopilot: Running AI Agents on Kubernetes With Open Source Tools
36mShift-Left for LLMs: Securing the AI Model Supply Chain