All of Our ML Ideas Are Bad (and We Should Feel Bad)
Talkat SREcon19 Europe/Middle East/Africa · on Wednesday, 2019-10-02· at 16:00· Track 2: The Liffey A·
summary
Most production ML proposals fail because the data and signal aren’t there: ticket prioritization and root‑cause models lack reliable labels, suffer delayed feedback and concept drift, and have far fewer examples than needed. Simple fixes—cut inbound ticket volume, add integration tests, use correlation and basic statistics—often outperform heavy ML. Plausible wins require hundreds of millions of datapoints, thousands to millions of labeled examples, and acceptance that many solutions are correlation not causation. Tools like TensorFlow add complexity, not magic.
AI-written summary · about
chapters (21)
- 0:00Intro
- 1:19Agenda
- 2:07Who am I
- 4:43Machine Learning vs AI
- 7:09Machine Learning Primer
- 14:38Deep Learning vs ML
- 15:06What is ML for
- 19:33Why ML is bad
- 20:08Ticket categorizing
- 20:49Single ticket queue
- 22:12Single ticket algorithm
- 24:55Whats wrong with this
- 26:53Automatic Root Cause Analysis
- 27:18Statistical Correlation
- 29:36Low Uncertainty
- 31:04Deep Learning
- 31:48Confirmation Bias
- 32:16Plausible Use
- 32:49Prework
- 33:21ontology epistemology metaphysics
- 34:00QA
related talks
21mWhy is Production ML so Hard?
25mAutomating Operations with ML
59mWhy Great Models Fail: Lessons From 9 Years of Deploying ML Models
27m10 Things That Can Go Wrong with ML Projects
32mWhy ML Projects Fail and How to Ensure Their Success
24mFoundations for ML at Scale: Where to Start in Your Organization
53mUnleashing the Black Box: Testing Machine Learning Systems
48mPrinciples and Practices of Building Machine Learning Systems