Data Cleansing & Beyond: How to Address Data Debt for AI
summary
Claim: poor data quality is one of the top two showstoppers for corporate AI and most teams waste effort cleaning downstream instead of fixing sources. Evidence: data scientists spend 60–70% of their time on prep, studies rank data quality high among adoption barriers, and ML projects fail at ~70–80%. Recommended actions: prioritize fix the source via database refactoring and upstream cleansing, choose techniques by five contextual factors (timeliness, automation, source impact, payback, skill), and treat data debt as the main technical debt to retire first.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 1:55AI and ML basics
- 3:20Data quality in AI
- 9:46ML lifecycle failures
- 19:13Data debt explained
- 23:26Fixing data at source
- 29:10Poor data consequences
- 35:35Data quality problems
- 39:31Choosing data techniques
- 48:00Summary
related talks
31mThe Hidden Costs of Data Quality - Tackling Common Data Challenges in ML
49mKeep it Clean: Why Bad Data Ruins Projects and How to Fix it
44mKeep it Clean: Why Bad Data Ruins Projects and How to Fix it
26mThe Hidden Foundation: How Effective Data Management Determines AI System Success
30mData Prep Ops: The Missing Piece of the MLOps Puzzle
25mImportance of Data Maturity before embracing AI
28mAI/ML for Non Data Scientists
39mDoing Data: The Critical Process of Data Preparation (...)