Modern Machine Learning flow with Quilt and Polyaxon
summary
Shows a concrete ML workflow: Quilt (S3‑backed data versioning) as an alternative to DVC for dataset packages, metadata and revision switching; Polyaxon on Kubernetes schedules hundreds+ experiments on Google Cloud, emitting CSVs and pickled model artifacts; CI via CircleCI and containerized builds with Docker for reproducible pipelines and deployment. Demonstrates a real app using Quilt for image datasets, describes jam/config files, resource limits, concurrency set to 20, and reports that the Polyaxon runs and artifact collection work end‑to‑end.
AI-written summary · about
chapters (6)
- 0:00Introduction
- 3:44ML flow challenges
- 7:03ML vs backend differences
- 10:43Data versioning with Quilt
- 22:05Polyaxon for experiments
- 30:42Q&A
related talks
38mAutomating Machine Learning and Deep Learning Workflows
19mEfficient Model Exploring and Continuous Delivery With Polyaxon + Kubeflow
33mCompliant Data Management and Machine Learning on Kubernetes
21mMastering the DevOps Machine Learning Pipeline
57mManaging Machine Learning Lifecycle with MLflow
01h 18mScaling and reproducing deep learning on Kubernetes with Polyaxon
18mFrom Silos to Synergy: MLOps & Developers Unified
30mHow to make continuous integration work with machine learning