Painless Multi-Tenant Kafka on Kubernetes with Istio at ASML
summary
ASML runs a vanilla Kafka cluster on Kubernetes using K operator and leverages Istio mutual TLS with SPIFFE workload IDs to give pods zero‑config auth. Two small plugins (an Envoy/WASM sidecar plugin and a Kafka auth callback) glue TLS identities into SASL, plus a custom operator (≈2,500 lines Go) reconciles Topic CRDs, TopicAccess CRDs and tenant quotas to provision Kafka ACLs and client‑quota callbacks. Result: multi‑tenant isolation, self‑service topics and working rate limits (example: 1 MB/s).
AI-written summary · about
chapters (11)
- 0:00ASML and team intro
- 1:05ASML’s semiconductor role
- 10:20Digital transformation challenges
- 16:20ASML’s digital platform overview
- 22:18Multi-tenant Kafka on Kubernetes
- 24:28Kafka on Kubernetes challenges
- 28:25Kafka client auth with Istio/SPIFFY
- 34:04Kafka topic access control
- 38:39Noisy neighbor quotas
- 43:35Final architecture & open source
- 45:04Q&A
related talks
47mPainless Multi-Tenant Kafka on Kubernetes with Istio at
42mPartial Multi-Tenancy on Kafka Using Strimzi at LittleHorse
23mManage Multi-tenant ML Workloads Using Istio
46mMulti-tenancy in Apache Kafka, Navigating a Major Topic
26mProduction Grade Kafka on K8s
33mKafka Operator: Managing and Operating Kafka Clusters in Kubernetes [A]
30mWho Are You? Configuring Kafka Authentication in Strimzi from TLS to Custom Principals
34mHow to Survive Managing Strimzi in an Enterprise