From Shadows to Spotlight - How Swiss Post Performs Reliable ML Deployment
summary
Swiss Post runs new fraud models behind a shadow variant that mirrors live traffic to a secondary SageMaker endpoint, stores inputs and predictions on S3, and measures latency, data/model/label drift and business KPIs. Synthetic requests carry expected labels to validate direction of drift. A Lambda + EventBridge flow promotes the shadow to production when weighted KPI thresholds beat the live model, then tears down the old variant. Training cost dominates, so shadowing is temporary and risk‑free.
AI-written summary · about
chapters (16)
- 0:00Reliable ML deployment intro
- 3:31Cyber crime risks
- 6:44Project IRIS fraud detection
- 7:50Automating ML training with SageMaker
- 9:13ML deployment architecture
- 13:51Manual deployment unsustainable
- 14:53Testing models before release
- 16:18Shadow model deployment
- 17:42Shadow deployment details
- 19:09Validating shadow models
- 20:34Model drift explained
- 23:23Synthetic data for drift
- 25:07Promoting models to production
- 27:15Academic to production
- 27:57Key takeaways
- 29:19Q&A on model switching
related talks
43mFrom Shadows to Spotlight - How Swiss Post Performs Reliable ML Deployment
37mThe story of IRES - How Swiss Post fights fraud with ML on AWS
35mHow Swisscom efficiently fine-tunes and deploys LLMs with Amazon SageMaker
34mLat Designing and Building Complex Machine Learning....
43mMachine Learning Ops: DevOps for Data Science
36mInformation Reliability Service - a novel approach to fraud detection
36mContinuous Monitoring of Machine Learning Models
21mZero Downtime ML Deployments