The Sanity Check: Observability for AI Data Pipelines
summary
Claim: attach provenance at ingestion so a bad LLM answer can be traced to the exact source. Shows an open spec called data bill of materials and an OSS project Makoto plus a commercial distributed pipeline Expanso node that annotates records with retrieval time, source and schema. Cites 175 zettabytes, 85% of AI projects never reach production and 60% of data outside DC. Demo ties a wrong “365 days” answer back to a Salesforce feed; demo mostly works after adding metadata.
AI-written summary · about
chapters (7)
- 0:00AI observability challenges
- 3:09Hidden data blind spot
- 8:07Data bill of materials
- 12:41Debugging AI pipeline failures
- 18:11Key takeaways for AI pipelines
- 21:01Observability prerequisite for trust
- 22:03Q&A on data problems
related talks
22mTHE SANITY CHECK: OBSERVABILITY FOR AI DATA PIPELINES
16mData pipeline automation with built-in metadata and lineage (PRT054)
19mObservability for Data Pipelines: Monitoring, Alerting & Tracing Lineage
44mData Quality Management
18mData quality: the key to long term happiness
14mData Lifecycle Management as Foundational Pillar to AI Agents
28mthe semantic layer your AI needs
29mData Observability: From 1788 To 2032