TGI Vault: Securing Kafka with Vault PKI
summary
A working three‑node Kafka cluster uses Vault PKI to mint node and client certificates and enforces mutual TLS for authentication while ACLs restrict topic access; clients fetch short‑lived certs (typical 1 hour client TTL) via Vault roles (Kafka_client, Kafka_server) and producers/consumers successfully exchange messages in the demo. Servers use longer certs and require rolling broker restarts to renew, so automated renewal monitoring is critical; code and a step‑by‑step blog are available on GitHub.
AI-written summary · about
chapters (14)
- 0:00Introduction and agenda
- 1:46Motivation for securing Kafka
- 5:17Kafka security options
- 7:23PKI fundamentals
- 9:35Vault PKI features
- 13:55Demo setup overview
- 17:52Certificate role restrictions
- 18:37Configuring Kafka ACLs
- 19:20Generating client certificates
- 20:50TLS mutual authentication test
- 21:53Multi-tenant PKI roles
- 23:42Certificate lifecycle management
- 25:47Monitoring and renewal
- 26:07Conclusion and resources
related talks
39mZero Trust RabbitMQ with HashiCorp Vault
41mKafka Security - A brief overview of its history ... #bbuzz
27mRunning Kafka in Kubernetes a practical guide
30mThe Secure Metamorphosis: Streaming Logs with Kafka and TLS
20mSecuring Kafka with SPIFFE at TransferWise
31mVault and Secret Management in Kubernetes
28mSecret Management with Hashicorp´s Vault
24mMake Your Kafka Cluster Production-Ready