Beyond unit tests
summary
Cisco Lysaker labs run a fully automated pipeline that builds each commit (about 70 artifacts, ~1 hour clean build, incremental faster) then books real hardware to run 600+ tests per commit across staged gates and nightly suites, with average tests taking 6–8 minutes and the system able to run ~800 in parallel; tooling is mostly Python automation with Jenkins, Kibana, Selenium and an internal LLY tools pixel compare flow, PDUs and USB-serial racks for device control, and event-driven APIs to detect regressions before shipping to ~100k devices.
AI-written summary · about
chapters (9)
- 0:00Introduction and organization
- 2:26Team roles and workflow
- 5:37Development and build pipeline
- 10:35Testing infrastructure
- 17:02Pipeline scheduling and scaling
- 22:23Quality and test motivation
- 24:43Tooling and API design
- 29:32Cultivating test culture
- 35:16Hardware scaling and labs
related talks
38mGrowing a Lab for Automated Upstream Testing: Challenges and Lessons Learned
56mTestOps at scale
32mBuilding Continuous Integration and Feedback
31m#testedAF: Test All the Things (Not Just Your Units)
33mBuilding an Automated Test System from Scratch
33mLKFT experiences and plans
41mThe magical fantasy land of Linux kernel testing
35mTest Harness: Continuous Testing on Hardware