GenAI Platform: Challenges of Authentication and Authorization in a...
summary
SAP runs an AI foundation with 40+ LLMs and sees real-scale failures: a Lua JWT filter jumps to 60 ms latency under load, sidecars swell to 16 GB/5 cores, and training workloads stream ~1 TB/day of logs. Fixes include replacing in-proxy Lua/WASM with an external Envoy processor (gRPC to native Go/Rust) for header and subscription lookups, moving auth to Open Policy Agent with caching instead of per-tenant auth policies, and shifting networking to ambient mode or Kubernetes-native sidecars plus taints/tolerations to eliminate init/race and model-download breakage.
AI-written summary · about
chapters (12)
- 0:00SAP AI platform intro
- 1:50AI foundation architecture
- 3:38Open-source components
- 6:06Istio ingress scaling
- 8:41JWT token extraction perf
- 10:51Tenant-specific JWKS at scale
- 12:59LLM model subscription lookup
- 15:13Logging under high load
- 17:00Istio race conditions
- 19:33KServe serving with Istio
- 20:56Lessons learned summary
- 22:44Q&A on mTLS and traffic
related talks
24mAI Governance for developers: A practical framework for deploying autonomous coding agents safely…
38mPanel Discussion: AI Needs a Foundation That Never Fails- 24mAI Governance for developers: A practical framework for deploying autonomous coding agents safely…
57mAI Applications Security Puzzle
23mCan AI Replace Authentication Systems? Frontline Security Lessons
14mBest Practices for Accelerating Your AI Journey
14mAI Sprawl: The Hidden Risk Threatening Your Enterprise Now
31mUser Account and Authorization Server at SAP