Lifecycle of Reusable Automations: Track, Maintain, Deprecate
summary
An in-house automation platform manages 2000+ automations and ~5 million executions per month by ingesting telemetry into a data lake, running SQL queries to derive success/failure, dependency and relevancy metrics, and surfacing them in the UI so operators see most-used automations and autosuggestions. The system pauses automations that fail integration tests, notifies contributors, and can auto-deprecate unused scripts after no response. It reports test coverage across fleet, requires six-month telemetry retention and resilient pipelines, and avoids ML—simple queries drive the lifecycle decisions.
AI-written summary · about
chapters (11)
- 0:00Intro
- 1:43Overview of Automation Platform
- 6:26Executing Automation
- 8:42Challenges that come with scale
- 11:18Case Study: Understanding script rot
- 15:45Use data points to decide automatically
- 16:36Defining lifecycle of an automation
- 18:12Data Collection
- 19:20Data aggregation and reporting
- 22:16Automation Flow
- 26:36Limitations
related talks
18mAutomation Lessons Learned
33mAutomating Data Analytics: Collect, Transform and Deploy - Lead With Change
18mAutomating Product Deprecation
53mAutomation Automation Automation
47mWhat are your Automation Strategies?
51mRedefining test automation
46mThe Practical Automation Playbook
38mScaling Automated Testing & Architecture from Startup to Enterprise