Our journey to horizontally scalabe controllers
summary
Hobbyfarm replaces a single monolithic controller with sharded, horizontally scalable controllers that use Kubernetes CRD informers, a simple fnv hash of metadata.name modulo replica count (example: four shards) to decide ownership, and per-replica work queues plus configurable worker threads. Replica identity comes from a StatefulSet identity and replicas watch that set to detect scaling; there is no leader election and no external message queue. Result: parallel, equally distributed reconciliation, faster startups, and lazy rebalance on next update.
AI-written summary · about
chapters (8)
- 0:00Introduction to scalable controllers
- 1:25Hobby Farm project overview
- 3:11Initial monolithic architecture
- 4:57Discovering microservices approach
- 6:44Introducing sharding concept
- 9:08Database sharding principles
- 12:19Final solution with sharding
- 17:20Q&A session
related talks
31mBeyond the Limits: Scaling Kubernetes Controllers Horizontally
43mDeep Dive: Apps SIG
36mBuilding a Fault-Tolerant Resources Controller on Kubernetes
19mSimplify Kubernetes Operator Development With a Modular Design Pattern
31mIt's all about reconciliation - anatomy of a kubernetes controller
27mDemystifying Why the World Is Built on Kubernetes: Learning To Leverage Bespoke CRDs and Controllers
10mTalk Deep Dive
46mOperators Live in the Past: Designing Reliable Kubernetes Controllers