Keynote: Fault Tolerant Machine Learning Operations
summary
Defines three maturity levels (0–2) and shows moving from ad hoc local models to automated retraining, architecture search and orchestration using DAGs and tools like Google Colab, AWS SageMaker and Metaflow. Recommends a Prototype Corpus for fast feedback, saving a 1% raw sample for 14 days while other data expires in 48 hours, daily distribution profiling, automated data validation, and UI feedback with 2–3 selectable responses. Discusses ensemble voting, lighter sequence‑to‑sequence plus regression pipelines, and cost‑aware training strategies.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 4:15Chelsea Troy background
- 6:40Machine learning operations
- 11:17Mozilla’s ML ops challenges
- 15:36ML maturity levels
- 27:17Prioritizing ML compute
- 31:58Model correctness in production
- 34:50Customer feedback and telemetry
- 38:31Summary
- 39:17Q&A
related talks
46mKeynote: How Can We Trust Machine Learning?
38mBuilding robust machine learning systems
28mMastering MLOps: Building Future Ready ML Systems
39mApplying deployment oriented mindset for building Machine Learning models
26mCompleting the machine learning loop
18mReliable Maintenance of Machine Learning Models
48mPrinciples and Practices of Building Machine Learning Systems
54mKeynote: Can you trust your (large language) model?