THE SANITY CHECK: OBSERVABILITY FOR AI DATA PIPELINES
summary
A chatbot answers “365 days” but the true policy is 30 days because a rogue Salesforce feed pollutes embeddings; the pipeline is a black box so top-line metrics look green while answers are wrong. The cure: add a data bill of materials (Makoto) attached at ingest, run schema checks and attestation, and gate embeddings so downstream training excludes records without lineage. Using an Expanso agent at edge nodes the team locates the exact source, refreshes embeddings, and fixes the answer end-to-end.
AI-written summary · about
chapters (10)
- 0:00Introduction to observability
- 1:12Why data pipeline observability matters
- 3:21Scale of the data problem
- 5:32AI pipelines complexity
- 7:16Missing data path observability
- 9:28Debugging without data traces
- 10:52Data bill of materials concept
- 13:18Distributed data pipeline demo
- 17:13Fixing the data pipeline
- 20:00Operating model for AI pipelines
related talks
27mThe Sanity Check: Observability for AI Data Pipelines
16mTaming Rogue AI Agents with Observability-Driven Evaluation
45mManaging Data & Analytics Projects Effectively
11m68% of enterprise AI fails here—here's how to fix it
25mThis Lying Has To Stop: Keeping AI Honest with OpenTelemetry
25mWhen Your "New Teammate" Hallucinates: Observability as the Safety Net for AI Code
21mGarbage In, Garbage Out: Elevating AI with High-Quality Data
14mData Lifecycle Management as Foundational Pillar to AI Agents