How Google Manages Sidecars for Millions of Containers Without Breaking Anything (Much

on YouTube ↗29m

" summary

Google replaces per-language client stacks by rolling a C++ sidecar to millions of processes, finishing a global rollout over about 18 months. Updates use a dev/stable release cadence (weekly dev, monthly stable), autopilot versus bound modes, restart-based deployments, per-team quick rollback buttons, and heavy performance work so CPU and latency stay tiny. Early experiments with shared memory failed and the system switches to domain sockets. A zero-day fix rolls out in days to a week without breaking user services.

AI-written summary · about

" chapters (12)

1. 0:00Introduction to sidecars
2. 0:38Why Google adopted sidecars
3. 3:23Overcoming operational challenges
4. 6:18Phased rollout strategy
5. 8:07Performance and communication
6. 10:43Managing the rollout pace
7. 11:41Upgrading and versioning
8. 13:56Rollback procedures
9. 16:02Debugging and lessons learned
10. 18:54Avoiding over-complication
11. 22:36Conclusion and results
12. 23:42Q&A session

" related talks

• 43mCloud Native - effective computing at scaleIgnacy Kowalczyk · GDG DevFest Sofia 2018
• 43mContainers Change EverythingAnne Currie · Codemotion Amsterdam 2017
• 11mAbstracting infrastructure away from developersPeter-Mark Verwoerd · O'Reilly Software Architecture Conference New York 2018
• 51mKubernetes: Changing the Way That we Think and Talk About ComputingBrian Dorsey · GOTO Berlin 2015
• 30mBuilding distributed systems is hard - case study (...)Natalia Sakowska · JDD 2018
• 57mDebugging Java in Production on Google Cloud PlatformPatrick Flynn · Devoxx US 2017
• 20mCollision 2015 - Urs Holzle of Google with Bruce Upbin of ForbesUrs Holzle, Bruce Upbin · Collision 2015
• 46mOk, so I have all these Containers, what now?Mandy Waite · GOTO London 2015