DataEngBytes 2023 - MEL-T1-05 - Izzudin Hafiz

on YouTube ↗24m

" summary

Claims that data teams commonly rely on Excel as primary catalog and that most teams cannot confidently answer "what data do we have" leading to lost trust; proposes a platform (DQ/Decube) that connects catalogs to sources, runs continuous declarative checks for volume freshness schema conformity and value distributions, and maintains automated lineage so classifications and impact analysis propagate downstream. Implementation goal is low-effort adoption and real-time visibility so incidents are detectable and root causes traceable.

AI-written summary · about

" chapters (12)

1. 0:00Introduction
2. 1:50Mid to large data teams issues
3. 4:19Data ecosystems complexity
4. 6:28Data cataloging solutions
5. 9:57Effective data catalogs requirements
6. 11:21Data observability
7. 13:29Data observability applications
8. 14:55Data trust importance
9. 16:43Implementing data observability
10. 17:47Data governance role
11. 20:34Data reliability and trust
12. 23:49Q&A

" related talks

• 36mDataEngBytes 2023 - PER-T1-02 - Josh ArmitageJosh Armitage · DataEngBytes Perth 2023
• 21mDataEngBytes 2023 - SYD-T2-04 - Suneeta MallSuneeta Mall · DataEngBytes Sydney 2023
• 27mDataEngBytes 2023 - BRI-T1-07 - Joe ReisJoe Reis · DataEngBytes Brisbane 2023
• 44mData MeshJochen Christ · KanDDDinsky 2022
• 20mDataEngBytes 2023 PER-T1-10 - Matthew CarterMatthew Carter · DataEngBytes Perth 2023
• 29mDataEngBytes 2023 - SYD-T1-08 - Rishu Saxena & John CosgroveRishu Saxena, John Cosgrove · DataEngBytes Sydney 2023
• 28mDataEngBytes 2023 - SYD-T1-03 - Chad SandersonChad Sanderson · DataEngBytes Sydney 2023
• 21mDataEngBytes 2023 - SYD-T3-04 - Abhinav GoyalAbhinav Goyal · DataEngBytes Sydney 2023