Not Omnipotent etcd, or Why It Doesn't Handle Heavy Loads
summary
The talk examines etcd’s limitations under heavy load, explaining why horizontal scaling of etcd clusters is problematic and why etcd isn’t suited for very large volumes (with experience cited from Google and AWS). It analyzes how the types and distribution of data stored in etcd affect performance, recommends what should not be kept inside etcd (to be moved outside the cluster), and shows how a single manifest error can push Kubernetes to its limits and break it.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 1:54Client history and early errors
- 5:10etcd issues at 5000 pods
- 6:33How etcd works and its limits
- 13:25Optimal master node count
- 15:27Second episode: 10 000 pods
- 17:02etcd variable overflow
- 22:41Third episode: 25 000 pods
- 26:21Summary and fixes
- 27:48Q&A
related talks
31mDemystifying Kubernetes Failure Scenarios Caused by etcd
15mKubernetes and etcd Features That Unlocked 15k Node Clusters
23mThe Secret Recipe of etcd
26mDon't Let Your Kubernetes Cluster Go Wild: Ensuring Etcd Reliability
20mLessons Learned from Operating ETCD
19mDr etcd; or; How I Learned to Stop Worrying and Love the Datastore - Nick Young, VMware
06m⚡ Lightning Talk: Back To Basics: How To Measure Etcd Performance And Not To Die Trying
06mKeynote: Distributed Systems Simplified Using Kubernetes