How We Used Statistics to Find Toil among 36,000 Changes
summary
Dataset of ~36,000 Terraform plans (saved as Terraform plans JSON) gets analyzed to flag routine changes, claiming 12–21% of PRs could bypass review. The pipeline uses a Pelt change-point detector to ignore bursty project work, computes two metrics—tasks saved and fractional share—to prioritize attributes, converts rates into a -5..+5 signal, then aggregates with a tunable alpha so one rare negative attribute can veto many green ones; examples: 21% saved via 8 resources, 12% via ~250 attribute approvals.
AI-written summary · about
chapters (10)
- 0:00Finding infrastructure toil
- 1:20Routine changes and bottlenecks
- 3:05Terraform plans for routine changes
- 5:38Metrics for auto-approval
- 7:24Pre-approval results
- 8:27Change point detection
- 10:52Traffic lights for signals
- 13:41Overall change assessment
- 16:30LLMs without math
- 18:18Q&A
related talks
57mUsing developer-centric data to predict, prioritize, and improve App Security Outcomes
32mFrom Toil To Triumph: Harnessing Agentic AI To Streamline Infrastructure as Code
17mTransforming API Management: A Case Study from Atlassian
26mOrchestrating AI Agents for Large-Scale Refactoring
23mFrom Blind Spots to Merged PRs: Continuous Agentic Performance Optimization
09mThe AI trick that cut AWS's issue triage time by 90% and caught regressions in 2 minutes
01h 01m"Looks Good to Me": A Practical Guide to Handling AI-Generated Code
26mYour pipeline is the bottleneck