Zero to production-ready: a best-practices process for Docker packaging
summary
Gives a prioritized, iterative process: get something running, then fix security, add automated builds, ensure reproducible pinning and only then optimize for build speed and size. Recommends running as non-root, embedding git commit via Docker labels, scanning with Trivy and Bandit, building in CI (GitHub Actions/GitLab/Jenkins), pinning deps with piptools, enabling the python fault handler, and using --no-cache-dir for smaller images — warns Alpine can be about 50x slower on builds for many Python wheels.
AI-written summary · about
chapters (23)
- 0:00Zero to production-ready: a best-practices process for Docker packaging Itamar Turner-Trauring
- 1:10complicated: Processes! • Packaging interacts with multiple organizational processes: Software development
- 2:24Today: learn a packaging process 1. You have limited time at work, can get interrupted at any moment 2. Many best practices. Which should you do first 3. You need an iterative Dockerizing processi Most important parts first. Each step builds on previous steps.
- 3:32A six steps packaging process 1. Get something working 2. Security
- 6:24Example: Listen on 0.0.0.0 . Problem: If you listen on 127.0.0.1, that's localhost inside the container . No one else can talk to a different computer container's localhost. • Solution listen on 0.0.0.8 so your server is accessible
- 7:44Security . Before you can deploy anything publicly, it needs to be secure . So we do that next
- 9:38Example: Update system packages
- 10:42Interlude: Packaging requires ongoing processes . Example best practices so far are done once. during initial packaging • Packaging isn't just about configuration files though • You also need ongoing processes
- 11:31Example: Security updates . Docker images are immutable artifacts, security updates require new image . Need ongoing process to: 1. Learn about security updates to dependencies 2. Update dependencies 3. Rebuild image 4. Redeploy application
- 14:00Integrating with your development process . Packaging interacts with how your team
- 14:43Example: Run security scanners . trivy can scan both Python dependencies and system packages for known security issues • bandit will find potential vulnerabilities in your code.eg SQL injection or use of pickle.
- 15:35Example: Tag based on branch . You want to build image for feature branch 123-more-cowbell automatically • You want production not to be impacted.
- 16:55Operational correctness and debuggability .Youve started automatically building and probably deploying • It's running in production! . Want it to run well in production • More likely to see errors, lots of images, need debugability
- 18:50Example: Make images identifiable Tags are not embedded in the image so can't be used to identify an image • Identity can be embedded using labels.
- 20:05Reproducibility . This week, your major dependencies probably won't change dramatically . Over six months, some of them will. • Over two years, most of them will. So next, you want reproducible builds so you can update in a controlled manner.
- 20:58Reproducibility as a process . Latest dependencies on every rebuild: No reproducibility • Thus: pin (freeze) your application dependencies (pip-tools/poetrypipenu). • If you don't update: obsolete dependencies. dangerous upgrades . Django LTS helps somewhat bute Dango 2.2 security updates end April 2022. Thus: need organizational process to update
- 22:22Example: Choose a stable base image . You'll want a Linux os which does security updates while still guaranteeing backwards compatibility for example Ubuntu LTS, Debian Stable, or Red Hat Enterprise Linux • The official python images are based on Debian Stable, but give access to newer Pythons.
- 23:215. Example: Pin Python dependencies pip-tools can turn this requirements. In
- 24:06Faster builds and smaller images . Your images are now packaged correctly, so now you can focus on optimizations. . Your time is expensive, you don't want to wait for builds • Smaller images can speed up test runs and production startup
- 24:37Example: Don't use Alpine Linux Alpine Linux can't use most precompiled wheels from PyPI. • As a result, you need to compile (almost) everything • Examples install pandas and matplotlib. python: 3.8-slim-buster: 30 seconds. python: 3.8-alpine: 1500 seconds, 50 slower!
- 25:51Example: Disable pip's caching . By default pip keeps copies of the downloaded package, in case you reinstall later, . This wastes space, increasing image size, and you won't need it
- 26:27Recap of steps 1. Get something working 2. Security
- 27:05Packaging isn't just about artifacts . Dockerizing requires artifacts: Dockerfile. builds scripts, etc. . But packaging also interacts with and requires processes! Interacts with development processes: version control branching, testing, CL Need process for security updates. Need process for dependence updates.
related talks
25m0 to production-ready: a best-practices process for Docker packaging
43mZero to production ready: a best practices process for Docker packaging
16mSimplifying the Journey of Your Containerized Application from Dev to Production
29m5 Security Best Practices For Production Ready Containers
30mPython Containers: Best Practices
24mSpeeding up Your Docker Image Build
51mProduction level Docker
40mDocker and Python: making them play nicely and securely for Data Science and ML