How We Built Out Our SRE Department to Support over 100 Million Users for the World's 3rd
summary
Aspiegel scales its SRE department from ~25 to over 100 engineers during 2020 to support Huawei’s mobile ecosystem (millions of users, 60 countries, 100+ services, tens of thousands of machines, petabytes of storage) while fully remote. Key artifacts: YSI CI/CD for remote deploys, AIOps monitoring for KPIs and SLOs, enforced one-plus-one redundancy for people and services, and dedicated IM support channels for tooling incidents. Hiring follows a planned timeline (expect ~3+ months to hire and ramp), onboarding uses four-week plans and mentors, a searchable knowledge base prevents tribal-knowledge loss, tooling capacity and license issues surface but are mitigated by the support channels, and the rebuilt org proves leaner and operationally stable.
AI-written summary · about
chapters (16)
- 0:00Intro
- 1:47Who We Are
- 2:56Our Setup
- 3:54What we will Discuss
- 4:52Friday 13th March 2020 - Ireland "Goes Home"
- 7:38Remote Tooling for the Remote SRE
- 9:25Choosing how to Communicate
- 13:00How to Grow the Team?
- 14:12Differences of a Small & Large SRE Dept
- 15:50Planning for Growth
- 18:08Hiring SREs in Bulk
- 20:22On-boarding Remotely, and at Scale
- 22:42The Next Stage - Knowledge Access
- 25:06Scaling up SRE Tooling
- 26:49Engineers Moving on
- 28:43So Where Are we Now?
related talks
43mSRE From Inception to Global Scale
30mThe Path to SRE
41mBuilding an SRE Capability Inside a Large Organization
26mFrom Nothing to SRE: Practical Guidance on Implementing SRE in Smaller Organisations
33mChallenges of Starting an SRE Team from Scratch in an Enterprise
18mDeploying Humans at the Edge of SRE
21mSRE: An incomplete guide to cultural Narnia
18mSRE Engagement Model Transition in Building and Expanding SRE Team