Snakes in a Buildpack: Deploying Data Science in Cloud Foundry
summary
Project ports Conda (via Miniconda) into the Cloud Foundry Python buildpack so binary-heavy PyData packages (NumPy, SciPy, OpenCV, scikit-learn, pandas) install without hours of C/Fortran compilation. Experiments that fail: adding libs to the rootfs (too large), vendoring Anaconda (huge). Final approach adds a separate Conda code path, fixes Miniconda forcing Python3, suppresses progress-bar log spam, enables app-cache reuse and pins Conda versions. Deployed to clients (delivery ETA, automotive supply/warning systems) and available in the official buildpack (v1.5.6).
AI-written summary · about
chapters (10)
- 0:00Introduction to Python buildpack
- 1:04Data science deployment challenges
- 3:51Why use buildpacks over Docker
- 4:54Exploring Conda support in buildpack
- 7:47Stabilizing Conda buildpack integration
- 9:55Real-world use cases and benefits
- 12:23Demo of Conda buildpack in action
- 16:46Future improvements for Conda buildpack
- 19:59How to contribute and get involved
- 23:36Q&A session
related talks
40mMaking packages and packaging "just work"
38mState of Data Science & Machine Learning
36mAnaconda Ecosystem for Open Data Science
45mConda: a cross-platform package manager for any binary distribution
37mEmbrace conda packages.
31mPython Anytime, Anywhere with Anaconda Notebooks
30mConda: a cross-platform, python-agnostic binary package manager.
30mEmpowered by Python - A success story