Jeremy Stanley, EVP/Data Scientist, Sailthru
summary
Sailthru launches Sightlines to predict per-user outcomes across ~5M users per client and ~1,000 clients, building ~10 models per client daily; the pipeline uses Amazon spot instances + Apache Mesos/Marathon and a custom relay allocator to reach ~90% utilization, claiming ~30x cost improvement versus reserved instances. JSON profiles are featurized with open source tidyjson, GBMs are distributed as many independent tree jobs, consistent 0.1% sharded sampling reduces I/O, and identical A/B environments enable safe weekly deploys.
AI-written summary · about
chapters (6)
- 0:00Introduction and business
- 1:28Technical architecture
- 11:58Resource allocation
- 12:39Modeling and data pipelines
- 21:21Deployment and maintenance
- 23:54Conclusion
related talks
17mDelivering ML-driven campaigns with RudderStack and Amazon Redshift
01h 13mDriving business with autonomous analytics and data democratization
40mReady, set, data: Perfecting the art of talent matching with AI
29mSeveral People Are Tuning: Data and Machine Learning at
41mPeer Beneath the Surface of the Earth
29mBuilding ML Platform Capabilities at Global Scale
17mCarlos Guestrin, CEO of Dato Inc. @ MLconf SEA
19mData science for real problems