Data Quality: The Holy Grail For A Data Fluent Organization
summary
A concrete data quality framework anchors on a retailer case: baseline schema rules up to fit for purpose rules for a dynamic pricing engine, with checks run at source, ingestion, transformation and consumption to shift left quality. Implementation uses domain-led domain data products and produces metrics (timeliness targets like one day, accuracy targets at ~90%) and automated rule checks (unit tests, aggregation reconciliations). The framework has run with clients and shows measurable trust gains and fewer downstream fixes.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 0:30Data Quality
- 2:50About Me
- 3:19Symptoms of Data Platform Failure
- 4:43Garbage in Garbage Out
- 5:27Case Study
- 8:30Causes
- 13:15Code vs Data
- 13:40Data Quality Definition
- 14:56Data Quality Dimensions
- 18:02Characteristics of Big Data
- 21:22Data Quality Framework
- 28:15Domain Data Products
- 33:48Source to Insight
- 37:05Continuous Data Quality Improvement Loop
related talks
16mShift Left Data Quality with Data Mesh Principles
37mData Validation at Scale: Managing Data Quality in Complex Data Pipelines
37mEngineering data quality
35mApplying Data Quality Monitoring
58mKEYNOTE: You Are Overthinking Data Quality
47mData governance
40mBuilding trust in your data. Why data governance is the key to success.
42mDeciphering the way data is tested