EVAPorating Kubernetes Security Risk: Adopting Validating Admission Policy at Scale

on YouTube ↗29m

" summary

Datadog replaces OPA Gatekeeper webhooks with ValidatingAdmissionPolicy evaluated in-process across 100+ clusters (10k nodes, 100k pods), translating a capabilities-restriction PSP into CEL expressions, a namespace-scoped parameter CRD, a reusable podSpec variable for deployments/jobs/pods, and user-facing message expressions. Migration uses audit mode, parses audit logs to match Gatekeeper denials, runs unit and E2E tests (dry-run on kind), and monitors CEL compilation/evaluation and the validation cost budget. Result: Gatekeeper is decommissioned and VAP runs in production with monitoring and a self-service exclusion workflow.

AI-written summary · about

" chapters (17)

1. 0:00Kubernetes security at Datadog
2. 1:47Kubernetes scale and policy needs
3. 2:51Migrating from Gatekeeper to VAP
4. 5:16Sample policy: tutorial to production
5. 10:31Policies with variables and parameters
6. 13:23Policy scope for multiple resources
7. 14:30Migration: Gatekeeper to VAP
8. 16:56Monitoring VAP performance
9. 19:46Policy evolution and benefits
10. 20:50Day-two operations and improvements
11. 23:41Q&A: Cluster resources in policies
12. 25:05Q&A: Policy behavior with audit logs
13. 25:50Q&A: Webhooks to VAP
14. 26:31Q&A: Testing ValidatingAdmissionPolicies
15. 27:13Q&A: Unit testing policies
16. 28:17Q&A: Client-side policy validation
17. 29:01Closing remarks

" related talks

• 28mEmbracing change: Policy-as-code for Kubernetes with OPA and GatekeeperAra Pulido · stackconf online 2021
• 34mGatekeeper: Flexible, Shareable Policy for KubernetesCraig Peters · KubeCon + CloudNativeCon China 2019
• 14mPolicy-as-code for Kubernetes with GatekeeperAra Pulido · KubeCon + CloudNativeCon Europe 2021
• 27mEmbracing change: Policy-as-code with OPA and GatekeeperAra Pulido · Conf42 Cloud Native 2021
• 34mEmbracing Change, Policy as code for Kubernetes with OPA and GatekeeperAra Pulido · DevOpsDays Portugal 2022
• 10mThe Power of OPA Gatekeeper: Enforcing Kubernetes Access Control to secure your runtimeBatel Zohar Tova, Dana Rozen · KCD Israel 2023
• 34mGatekeeper: Flexible, Shareable Policy for KubernetesCraig Peters · KubeCon + CloudNativeCon + Open Source Summit China 2019
• 33mBringing Pod Security Admission to Your Kubernetes DistributionStanislav Láznička · KCD Austria 2023