Kafka Operator: Managing and Operating Kafka Clusters in Kubernetes [A]
summary
Deploys Kafka and Zookeeper as StatefulSet + headless service to preserve stable IDs and DNS, schedules brokers with nodeSelectors and anti‑affinity onto SSD-backed storage using volumeClaimTemplates or local emptyDir for performance, runs via Helm/OpenShift templates, exposes metrics via JMX/Prometheus. An operator watches ConfigMap-as-topic descriptors and automatically creates/deletes topics and assigns ACLs, generating credentials so operator creates secrets per deployment. Demo shows topic creation and credential assignment succeed; upgrades and cross‑cluster blue/green migrations remain manual and tricky.
AI-written summary · about
chapters (23)
- 0:00Introduction
- 0:10Who are you
- 1:45What is a Broker
- 2:29Kafka in Kubernetes
- 4:27Kafka Zookeeper
- 5:13Kubernetes Deployment
- 5:49Demo
- 7:38Kafka Deployment
- 8:58Kafka Persistent Storage
- 9:31Kafka Persistent Volumes
- 11:35Monitoring Kafka
- 11:59Prometheus
- 12:27Demo Power
- 14:46Operators
- 15:26Provisioning
- 16:15Topics
- 17:30Config Maps
- 18:18Config Map Demo
- 19:06Security
- 20:05Operator logs
- 21:21Kafka upgrades
- 22:56Kafka performance
- 24:29Questions
related talks
29mKafka on Kubernetes: Does it really have to be the hard way?
26mProduction Grade Kafka on K8s
50mRunning Apache Kafka on Kubernetes and OpenShift
53mAutomating Stateful Applications with Kubernetes Operators
40mKafka, DevOps...And Resilience for All
27mRunning Kafka in Kubernetes a practical guide
44mTaming Apache Kafka Using GitOps
24mKafka & Kubernetes - Migrating to a World Without Zookeeper