Why Most Data Projects Fail & How to Avoid It
summary
Claims that 85% of data projects fail and that LLMs like GPT-3/GPT-4 aren’t a silver bullet, with concrete fixes: pick architectures after use cases, avoid vendor-driven designs (Cloudera cited), deliver in tranches, and staff at about 5:1 data engineers to data scientists instead of 1:10. Expect cloud spend tradeoffs (example million euros vs 200k) and aim for a 10x ROI target. Includes references to Flink vs Spark decisions and the O’Reilly piece on complexity in big data.
AI-written summary · about
chapters (13)
- 0:00Intro
- 1:4485% of data projects fail
- 2:53The right questions
- 3:19The right people at the right ratios
- 6:16Data teams
- 7:04What is the business value?
- 10:41When will you generate value?
- 13:43Where will data be processed?
- 17:06How will the plan be executed?
- 19:31Why is data valuable?
- 22:25Not all gaps are technology
- 24:19Should you get help?
- 26:17Outro
related talks
22mWhy Most Data Projects Fail & How to Avoid It
20mWhy AI Projects Fail: The Missing Link of a Strong Data Strategy
31mThe State of Data Engineering
44mStreamlining Your Approach to Data
26mThe Hidden Foundation: How Effective Data Management Determines AI System Success
27mWhy A Truly Great Data Strategy Isn’t About Data
33mPanel Discussion
59mThe Great Data Debate