Data Science Best Practices : From Proof of Concepts to Production
summary
Claims that 87% of data science projects fail and that the bottleneck is organization and integration rather than modeling. Shows concrete pipeline choices: prototyping in Jupyter pandas scikit-learn → production via Airflow PostgreSQL Hadoop, a common team makeup of two data scientists plus one data engineer working in 30–40 day sprints, and real pitfalls — e.g., EC2 cost tradeoffs and AWS Lambda memory limits forcing model rework when moving from POC to production.
AI-written summary · about
chapters (16)
- 0:00Intro
- 0:19Vision
- 0:57Modeling vs Production
- 2:27DJ Patil
- 3:16Kaggle
- 7:12Efficient Data Science Project
- 8:09Good Data Science Project
- 9:49Business Problem
- 12:02Using the Right Data
- 13:27What are the Right Models
- 13:37Interpretability
- 16:58Methodology Organization
- 17:18Global Vision
- 18:17Tools Infrastructure
- 18:57Process Flow
- 27:23Reusable Code
related talks
25mBuilding A Successful Data Science Team
28mThe Coding Portion of Data Science
36mFrom POC to Prod, a data science project laid bare
32mWhy Most Data Projects Fail and How to Avoid It
54mFrom the Life of a Data Scientist
59mDevOps for Data Science
17mProductionizing Data Science Workloads
43mAgile Strategies for High-Impact Data Science & Machine Learning Projects