Open Source DataOps and MLOps Strategies
summary
Outlines a practical path for open source DataOps and MLOps that centers on measurable wins: prioritize four domains—accuracy and data quality, visibility and lineage, reliability and latency, and security—and instrument pipelines with versioning, metadata, and contracts. Recommends tools and patterns: use Great Expectations, dbt, Airflow, DVC and gitops for tests and reproducible deployments, plus a metadata lake like gravitino for cross‑cloud discovery. Emphasizes ownership, incident playbooks, and incremental investment to avoid vendor lock‑in.
AI-written summary · about
chapters (15)
- 0:00Open Source DataOps & MLOps intro
- 1:42DataOps & MLOps frameworks
- 3:28Modern DataOps needs
- 5:53DataOps challenges
- 7:18Four DataOps strategy domains
- 9:26DataOps vs DevOps vs MLOps
- 11:29Pitching DataOps
- 12:31Five data observability pillars
- 14:41Data quality & testing tools
- 15:25Metadata in DataOps
- 17:11Version control & data contracts
- 18:16Incident management in pipelines
- 19:21MLOps practices overview
- 21:06MLOps roadmap & frameworks
- 22:30Q&A
related talks
29mDevOps for Data Science? - automate the boring stuff and leverage the OSS ecosystem
37mWhat is DataOps? Becoming a Truly Data-Driven Organization
14mWhy Your DevOps Skills Are Your MLOps
30mDevOps For Data Teams
43mNavigating the MLOps Journey: Key Considerations for Successful Implementations
01mMLOps: Applying DevOps to ML
13mMLOps at Scale
18mFrom DevOps to DataOps - Cloud based Software Development