Data Ecosystem to power AI
summary
Retail site with 59 million monthly visits and 8 million SKUs uses an event-first design that publishes backwards-compatible domain events from each microservice so source systems own state and downstream systems consume the same canonical facts. The pipeline feeds a feature store for serving and training and a single consolidated event stream that unifies batch and real-time processing, cutting replication, preserving semantics for reporting and ML, and reducing integration toil and time to production.
AI-written summary · about
chapters (11)
- 0:00Intro
- 1:32Service Oriented Architecture
- 4:213NF databases for BI
- 5:57Machine Learning Production
- 7:33Merging the 3 patterns
- 11:06Change in architecture
- 11:48Characteristics of Microservices built with Reactive Principles
- 14:04Event-First Domain Driven Design
- 16:42[Reporting Featurization Alerting] as outputs of Event processing
- 17:35Batch vs real time distinctions
- 19:41Outcomes
related talks
48mFrom (Big) Data Mess to Data as an Innovation Enabler
24mRetail Transactions at scale for AI - From PoC to production
01h 17mData Engineering in the AI Era: Building a Robust Foundation for AI Applications
04mdata science architecture
57mOur DataMesh Journey: data architectures for microservices
49mTransforming Analytical Data with Event-Driven Architecture
18mBehavioral Data Creation for AI - Delivering Value and Insights
31mOne Data Pipeline to Rule Them All