Did Kubernetes Make My p95s Worse?
Talkat KubeCon + CloudNativeCon North America 2019 · on Wednesday, 2019-11-20· at 11:50· Room 15AB - San Diego Convention Center, Mezzanine Level·
summary
Airbnb finds Kubernetes itself is not the sole culprit but several concrete factors raise p95s: migrations often change hardware generations so performance can improve or degrade; set CPU limits because unbounded containers create throttling spikes that standard metrics miss; image-locality and scheduler scoring pile 18 identical pods onto one host, consider pod topology spread constraints; Java JVMs that aren’t container-aware blow DB latency until teams upgrade JVM; iptables service balancing and kube-dns randomness hit an 80 qps limit—move to cloud provider native IPs or Envoy and adjust DNS policy.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 3:18Problems
- 4:18Case Study
- 6:42Noisey Neighbors
- 16:52WriteOnceRun Anywhere
- 20:44Traffic and Balance
- 24:44Keep DNS Slow
related talks
35mKubernetes, Lessons Learnt
36mKubernetes Confessions: Real Mistakes We’ve All Made (and What They Teach Us)
33mThe Big Kubernetes Mistake Paying More Than Necessary
24mPropagating Programming Paradigms: Lifting High Performance Compute Into K8s
47mSecrets of Performance Tuning Java on Kubernetes
27mRunning Kubernetes in Production A Million Ways to Crash Your Cluster
20m10 Common Things People Do Wrong in K8s Environments
32mOptimizing Application Performance on Kubernetes