Applied Data Science
summary
Claims that production fails for lack of engineering detail not theory and shows concrete cases: a Spark pipeline goes from 2 to 20 hours when a random forest depth from 10 to 15 makes the model size 30MB to 250MB so every partition reloads the model; a predictive maintenance model flags brake failure weeks early but misses most incidents because data arrives with a two week lag; a single data feed costs 300 hours/year in manual emailing; emphasizes data contracts and SLAs and production-quality code over notebook proofs.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 3:19Machine learning beyond accuracy
- 6:32Modern data pipeline challenges
- 9:49Spark performance pitfalls
- 13:26From prototypes to production
- 17:50Data automation and feedback
- 20:04Measuring success beyond accuracy
- 23:44Data contracts and SLAs
- 26:35Skills for production data science
- 30:33Building production systems
- 32:18Software for data products
- 34:48Closing remarks
related talks
28mThe Coding Portion of Data Science
38mData Secrets from a Platform Engineer
39mRight Code, Right Place, Right Time
30mFailing Fast with Data: Bringing CI to the Land of Data
22mBuild Smarter, Don't Just Build
37mBridging the gap: from Data Science to service
40mBuilding Successful Data Science Projects
41mBetter Code for Data Science