Enhancing Kafka Topic Management in Kubernetes With the Unidirectional Topic Oper...
summary
Unidirectional Topic Operator (UTO) replaces the old bidirectional model, switching to unidirectional reconciliation from Kubernetes to Kafka, becoming stateless and scaling far better (new GA in v0.41). It batches topic events, supports KRaft and TLS/SASL, adds finalizers and per-operation metrics, and integrates with Cruise Control to perform safe replication‑factor changes (demo moves a topic from 1 to 3 replicas). Operator runs standalone or embedded, and replication changes execute asynchronously via Cruise Control.
AI-written summary · about
chapters (25)
- 0:00Introduction
- 0:22What is Kafka Topic Operator
- 1:05Kafka topics and partitions
- 3:34Key partition configs
- 6:05Replication factor and ISR
- 7:09Retention policies
- 8:59Message ordering guarantees
- 10:06Managing Kafka topics challenges
- 12:31Monitoring Kafka with Grafana
- 12:54Introducing Topic Operator
- 13:36Topic Operator deployment modes
- 15:23Topic Operator security
- 15:44KafkaTopic custom resource
- 17:50Topic Operator history
- 20:40New unidirectional architecture
- 21:22Key improvements
- 22:52New Topic Operator features
- 24:56Old generation issues
- 26:25Performance comparison
- 27:30New Topic Operator features
- 30:25Replication factor change
- 32:32Demo: Replication factor change
- 36:27Multinamespace reconciliation
- 39:01Risk conditions
- 40:46Useful resources and documentation
related talks
33mKafka Operator: Managing and Operating Kafka Clusters in Kubernetes [A]
26mProduction Grade Kafka on K8s
31mEvent driven communication with Kafka on Kubernetes
44mTaming Apache Kafka Using GitOps
24mKafka & Kubernetes - Migrating to a World Without Zookeeper
29mKafka on Kubernetes: Does it really have to be the hard way?
46mMulti-tenancy in Apache Kafka, Navigating a Major Topic
50mRunning Apache Kafka on Kubernetes and OpenShift