Partial Multi-Tenancy on Kafka Using Strimzi at LittleHorse
summary
LittleHorse runs many tenant servers on one shared Kafka cluster using Strimzi CRDs to automate isolation: topics are created via KafkaTopic CRD, clients get TLS identities and prefixed ACLs via KafkaUser CRD, and cluster rebalancing uses KafkaRebalance (Cruise Control). Quotas are applied per broker (cluster quota ≈ per-broker quota × broker count), so the operator overprovisions during rebalances, then reduces limits after rebalance completes. An internal operator auto-scales node pools, triggers rebalances and updates quotas; the flow is implemented and works in production.
AI-written summary · about
chapters (10)
- 0:00Partial multi-tenancy with Strimzi
- 1:04Little Horse workflow engine
- 3:29Trade-offs for shared Kafka
- 6:19Topic management with KafkaTopic
- 10:11Encryption and access control
- 17:00Least privilege with ACLs
- 22:36Noisy Neighbor problem and quotas
- 26:31Balancing cluster with Cruise Control
- 33:55Custom resources in practice
- 37:28Closing notes and Q&A
related talks
32mStretching Strimzi: Exploring Multi-Cluster Kafka for High Availability
46mMulti-tenancy in Apache Kafka, Navigating a Major Topic
31mSimplifying Apache Kafka on Kubernetes With Strimzi
30mWho Are You? Configuring Kafka Authentication in Strimzi from TLS to Custom Principals
29mMastering Kafka Workload Balancing With Strimzi’s Cruise Control Integration
30mFrom Running to Operating: Real-World Strimzi in Production
47mPainless Multi-Tenant Kafka on Kubernetes with Istio at
26mProduction Grade Kafka on K8s