Chris Stucchio of Simpl at Machine Learning Developers Summit 2019
summary
Argues that production ML needs engineering rigour by showing a pay-later stack where a fraud model with ~50% recall and 1% false positive on good users yields ~17% false positives, then a cashback campaign multiplies good traffic by five and pushes the 50% false positive rate threshold that trips a circuit breaker and disables fraud. Systems run as Kafka event pipelines, models deployed as random forest/decision-tree code, and cascading model inputs break underwriting and retention; recommends continuous monitoring and paying to collect real counterfactual data.
AI-written summary · about
chapters (14)
- 0:00Start
- 3:41Accounting Identity
- 4:53Identify the Unknowns
- 6:40Good Code and Good Engineering
- 9:56Data Science System
- 10:59Cash Backs
- 11:53Circuit Breakers
- 16:20How Many Times Do You Eat Lunch per Day
- 19:00The Single Responsibility Principle
- 19:28Stop Fraud
- 20:03Underwriting
- 24:05Retention Model
- 27:23Selection Bias
- 32:35Continuous Monitoring
related talks
13mHarnessing Machine Learning to Combat E-commerce Fraud
16mLeveraging ML to Create a Reliable Anti-Fraud System
37mThe Role of Machine Learning in Today’s Payment Ecosystems
56mData, AI, Hype and Fear - Recipe for Disaster?
32mMaking Models Work: How an optimized ML lifecycle can drive better ROI
16mOperationalizing AI in Finance: Real-Time Analytics
47mTrain, Evaluate, Repeat: Building a Credit Card Fraud Detection System
13mHuman-in-the-Loop MLOps: Boost Model Performance by 40%