CloudLobster — the future of CI/CD management
summary
CloudLobster is an API that provisions per-team Jenkins on AWS by calling CloudFormation to create load balancers, ASGs, IAM roles, S3 buckets and credentials so teams get a fully configured CI server in minutes. Prototype with EC2 autoscaling proved insufficient at scale — Jenkins queues peaked at ~8,000 jobs — so the current system runs over 50 Jenkins masters and a few hundred EC2 workers managed by a second service, CloudOctopus slave API, implemented in Java 8 with Spring and the AWS Java SDK and planned for open source on GitHub.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 0:30Pawels background
- 1:32What is Zalando
- 3:00Technical difficulties
- 7:00Old setup
- 8:09Why not scaling
- 9:32AWS cloud implementation
- 10:04CloudLobster setup
- 10:47CloudLobster API
- 12:16CloudLobster overview
- 13:34Open source
- 14:20Continuous integration
- 15:10Slave overload
- 15:49Jenkins plugin
- 16:54Stickers
- 17:35Questions
related talks
08mjclouds & Jenkins: Any Cloud, Anywhere
22mAll Things Jenkins and Cloud Foundry
42mAutomation, Innovation and Continuous Delivery
38mOperating Jenkins at Scale
35mJenkins Scaling and Organization for an Efficient CI
15mJenkins on AWS for both small teams and enterprises
40mIntegrated Pipeline for Private & Public Clouds w/Jenkins, &
47mJenkins in the Enterprise: Building Resilient CI Infrastructure