From Frankenstein To Kamaji: Lessons in Building a Single CAPI Cluster Across Multiple Providers
summary
Built a hybrid workload cluster with control plane VMs on vSphere and bare‑metal workers on Proxmox, which triggers a deletion loop because the vSphere cloud‑controller manager (CCM) removes nodes it cannot find. Fixed locally by forking and patching the vSphere CCM patch to ignore nodes by providerID prefix, but this creates update and on‑call risk and the PR is not upstream. Replaced the VM control plane with Kamaji hosted control planes running as pods in the management cluster backed by etcd/MySQL, reducing VM waste but increasing MC blast radius and resource/backup complexity.
AI-written summary · about
chapters (13)
- 0:00Speakers and Cluster API intro
- 6:57Hybrid Kubernetes with Cluster API
- 9:08Dual-provider workload cluster attempt
- 12:41Live demo: failed dual-provider setup
- 13:23Why dual-provider setup failed
- 14:49Patching vSphere CCM for mixed providers
- 17:02Costs and limits of Frankenstein’s cluster
- 17:47Introducing Kamaji hosted control planes
- 20:15Building Kamaji workload cluster
- 21:44Live demo: Kamaji workload cluster
- 23:50Kamaji trade-offs and challenges
- 25:15Why CAPI should support mixed providers
- 26:02Q&A
related talks
37mFrom Frankenstein to Kamaji: Lessons in Building a Single CAPI Cluster Across Multiple Providers
40mMay the Control Plane be with you: Kamaji and the rise of Kubernetes at scale
35mKubernetes-in-Kubernetes with Kamaji, KubeVirt, and the Cluster API
26mRunning Kubernetes Clusters the Kubernetes-Native Way
35mSimplified Experience Of Building Cluster API Provider In Multitenant Cloud
22mTaming the VM Sprawl: KCP Meets KubeVirt
37mKubeInception: running up to a thousand clusters in the real Cloud-Native way
37mVirtual Clusters in the Real World: Lessons from k3k’s Journey to Production