The industrial challenge of missing data
summary
Overview covers origins of missingness and concrete Python implementations, showing scikit‑learn throws errors on NA while LightGBM and XGBoost handle missingness differently. Demonstrates single vs multiple imputation (mean/median, k‑NN, MICE), discriminative and generative methods, a denoising autoencoder and GAIN, with reproducible Colab notebooks and an image experiment with 20% missingness. Finds autoencoder and GAIN beat simple mean imputation and warns about dangerous package defaults like zero‑replacement.
AI-written summary · about
chapters (9)
- 0:00Missing data challenges
- 1:08Missing data industrial problem
- 3:15Types and origins of missingness
- 8:51Handling missing data strategies
- 15:10Exploiting target data for imputation
- 17:16Training models with incomplete data
- 19:00Imputation techniques demo
- 20:48Key takeaways
- 22:12Q&A on bias and noise
related talks
47mA Python Package for Grappling with Missing Data
06mThe Future of Missing Data
30mMachine Learning Techniques for Class Imbalances & Adversaries
48mCommon Data Science Mistakes
39mBeyond Bias: Crafting Fairness in Pythonic Machine Learning Systems
18mThe Importance of Modeling Data Collection
35mMental Models to Use and Avoid as a Data Scientist
45mA Practical Introduction to Data Science