Managing Changes Seamlessly on Yahoo's Hadoop Infrastructure Servers
summary
Yahoo runs Hadoop on ~45,000 machines and pushes ~15 changes per week across OS, Hadoop and infra. Automation enforces one-click deploy from sandbox → research → multiple production clusters, with Chef, SSH, Yahoo Vines installer, rolling/component-safe upgrades, and Jenkins validations. Deploys target components or a random percentage of nodes, use fail-fast checks, roll-forward with ability to rollback, and log every change to Elasticsearch plus a pipeline visualizer and grid auto-brake-fix for immediate remediation.
AI-written summary · about
chapters (19)
- 0:00Introduction
- 0:43Hadoop Stack at Yahoo
- 1:10Hadoop at Yahoo
- 1:40Changes
- 2:57Rules
- 4:35Agenda
- 5:35Tools
- 7:09Promotion Gates
- 8:25Grid Pipeline
- 11:53Promotion
- 13:21Conflicts
- 14:45RPM Package Management
- 17:45Component Safe Upgrade
- 19:55Monitoring
- 20:37Pipeline Visualization
- 21:09Conversion Visualization
- 21:43Change Metrics
- 22:44Validation
- 24:23Summary
related talks
40mChef @ Yahoo! JAPAN: How Chef Powers the...
32m“The Yahoo! Story” How Yahoo! Embraced Automated Testing to...
29mGetting Faster Answers at Yahoo Answers
16mCase Study: How Yahoo! JAPAN Uses and Contributes to Screwdriver at Scale
15mYahoo Case Study: Automated Canary Analysis using Spinnaker Kayenta
32mWriting Performance User Plugins for an HA Environment
46m10+ Deploys Pe
44mAutomating a Big Data Platform with Jenkins