Reproducible Machine Learning at Scale

on YouTube ↗30m

" summary

Shows how teams use Weights & Biases to track datasets, version training recipes, and store run metadata so experiments are auditable and shareable; demonstrates a deployed example from Blue River where camera-driven weed-targeted spraying cuts herbicide use dramatically; reports a Kaggle search-relevance lift from 35% to 60% then quick plateau; highlights reproducibility killers like glob non-determinism and GPU nondeterminism, argues reproducibility + lightweight integrations beat full autoML, and describes a working developer-first MLOps stack.

AI-written summary · about

" chapters (39)

1. 0:00Intro
2. 1:53Lessons from real world ML projects
3. 2:25My first ML stack
4. 3:02Working at Yahoo
5. 4:42Speech Recognition
6. 5:54Weed Detection
7. 7:41Healthcare
8. 8:51Robotics
9. 9:15Medical Imaging
10. 10:00Autonomous Vehicles
11. 10:28How is ML different than Engineering
12. 10:42ML is a subfield of Engineering
13. 11:05ML requires fundamentally different processes
14. 12:04Why not use git for version control
15. 13:36Debugging is more than engineering
16. 14:13Explainability
17. 15:45Debugging
18. 16:41Bugs
19. 17:43Whats hard and easy
20. 19:01Kaggle
21. 20:18Exponential curves
22. 21:10Reproducible ML
23. 21:34The reproducibility crisis
24. 22:18Checklist
25. 22:34Scale increases
26. 22:54What do we do
27. 23:17Resource consumption
28. 24:05Dependency graph
29. 24:32Transfer learning
30. 24:42Performance
31. 24:55Prompt Engineers
32. 25:15Responsible AI
33. 25:54Why Weights Biases
34. 26:22The Goal of Weights Biases
35. 26:38Other Approaches to ML
36. 27:09Premium Model
37. 27:24Interoperability
38. 27:37Integration
39. 28:40Learning ML

" related talks

• 30mReproducible Machine LearningMateusz Opala · PyData Warsaw 2019
• 01h 16mReproducible machine learning and science with pythonPrabhant Singh · PyCon DE & PyData Berlin 2022
• 30mRealizing End to End Reproducible Machine Learning on KubernetesSuneeta Mall · KubeCon + CloudNativeCon North America 2019
• 29mReproducibility in ML Systems: A Netflix OriginalFerras Hamad · PyData New York City 2019
• 20mBetter Care and Feeding of Machine Learning ModelsJeanne Choo · PyCon APAC 2018
• 28mTrusted AI – Building Reproducible, Unbiased and...Romeo Kienzler · PyData Warsaw 2019
• 25mBuilding Scalable MLOps Pipelines That Actually WorkSrikanth Vissarapu · Conf42 MLOps 2025
• 28mUnravelling Hidden Technical Debt in ML: A Pythonic Approach to Robust SystemsRavi Singh · PyData Global 2023