Scaling Your Jenkins Jobs
summary
OpenStack CI runs thousands of Jenkins jobs—roughly 4,000–6,000 jobs across 10 masters with ~100 slaves each and about 24,000 jobs per day—and managing that scale uses a Python command‑line tool, Jenkins Job Builder. Job definitions live in YAML templates that JJB converts to Jenkins XML, reducing thousands of divergent XML files to 87 JJB files that generate ~6,000 jobs, integrate with Puppet for automated updates, and operate idempotently in production.
AI-written summary · about
chapters (24)
- 0:00Introduction
- 1:21What is OpenStack
- 3:17Tools we use
- 5:26Workflow
- 6:58CI Workflow
- 8:22CI Scale
- 9:53Pipelines
- 12:55Job Growth
- 13:28Problems with Jenkins
- 14:26Jenkins Job Builder
- 15:56Jenkins Gel Builder
- 16:41Job Builder
- 19:18Update Jobs
- 20:53Job Definition
- 23:15Templates
- 24:56Running Jenkins
- 26:44Using Jenkins in XML
- 28:14Jenkins Job Builder Benefits
- 30:56Other Jenkins Job Builder Features
- 32:48Jenkins Job Builder Demo
- 33:24Jenkins Unit Test
- 33:55Future Improvements
- 35:09References
- 35:44Questions
related talks
35mJenkins Scaling and Organization for an Efficient CI
41mScaling Jenkins
15mAuto-scaling Jenkins Cluster on OpenStack Cloud Platform
22mAll Things Jenkins and Cloud Foundry
55mContinuous Integration on Steroids
38mOperating Jenkins at Scale
42mSeven Habits of Highly Effective Jenkins Users
37mHelping Enterprises Scale Their Jenkins Software Practices and Tools