Admission Webhooks: Configuration and Debugging Best Practices
Talkat KubeCon + CloudNativeCon North America 2019 · on Tuesday, 2019-11-19· at 15:20· Room 29ABCD - San Diego Convention Center, Upper Level·
summary
Admission webhooks graduate to GA in Kubernetes 1.16 and are powerful but brittle: mutating webhooks can be invoked twice so make them idempotent mutating webhooks, set matchPolicy to Equivalent to catch all API versions, and use short timeouts (example given 2s) and explicit failurePolicy (Reject or Ignore). Debug with the API server metrics (admission_webhook_admission_duration_seconds and rejection counters) and audit log annotations (the mutation and patch keys) to see which webhook patched an object; avoid side effects and exclude kube-system to prevent deadlocks.
AI-written summary · about
chapters (22)
- 0:00Intro
- 0:17Agenda
- 1:49Admission Webhooks
- 2:19Why do I need admission webhook 2
- 3:11Configuration fields
- 4:08Best practices
- 7:03Example of idempotent webhook
- 7:18Example of non-idempotent webhodk
- 7:52Intercepting all versions of an object
- 8:58Availability
- 9:44Guaranteeing the final state of the object is soll
- 10:26Side effects
- 11:28Avoiding operating on the kube-system namespace
- 13:36Types of webhook failure/rejection
- 16:57Metrics
- 18:41Besides monitoring
- 19:33Audit Logging
- 20:57Audit Record for Mutating Webhooks
- 22:11Patch occurrence
- 23:23Patch mutation
- 24:51Debug the webhook backend
- 30:27Key Takeaways
related talks
25mCreating and Debugging Kubernetes Admission Webhooks
20mUsing, and mis-using K8s Dynamic Admission Control
35mEnforcing Bespoke Policies in Kubernetes [I]
13mWebhook Topology and Admission Latency: Lessons from Migration
30mSupporting Long-Lived Pods Using a Simple Kubernetes Webhook
35mHow to fight misconfiguration and bad intentions with Kubernetes admission policy
38mFuzzing and Testing the Security of Kubernetes Admission Controllers
29mEVAPorating Kubernetes Security Risk: Adopting Validating Admission Policy at Scale