Masterclass: Building a data stack in 2021
summary
After 30 years of tooling the modern data stack crystallizes into specific choices: use ELT connectors like Fivetran (or open source Airbyte/Meltano) to load APIs and Shopify/GA into a single analytics warehouse, transform with dbt, and pick Snowflake (performance and data sharing favored over BigQuery/Redshift). Add a metrics layer (Transform, Supergrain or Looker) to enforce consistent definitions, push enriched scores back via reverse ETL to Salesforce or Sheets, and defer full ML production because MLops is still immature.
AI-written summary · about
chapters (20)
- 0:00Introduction
- 1:23Data AI landscape
- 1:59Questions
- 5:19Flowchart
- 10:08Speed
- 11:06Thirdparty platforms
- 15:59Event tracking
- 19:20Data warehouses
- 24:05Data transformation
- 25:05DBT
- 29:20Conception
- 31:45Complexity
- 32:25Metrics
- 34:20BI tools
- 36:05Customer analytics
- 37:45Other modalities
- 39:35Monitoring
- 40:25Reverse API
- 43:05Machine Learning
- 44:10Machine Learning and AI
related talks
43mData Modelling for Startups: Domain Models & the Modern Data Stack
25mPerfecting Our Modern Data Stack Recipe: What We Learned On The Journey At Huel
38mThe Rise & Downfall of the Data Engineer REVISITED
01h 35mThe Modern Data Stack
41mModern Data Architectures – What They Are and How to Implement?
01h 17mData Engineering in the AI Era: Building a Robust Foundation for AI Applications
01h 28mA Guide to the Modern Data Stack
11mRecent trends in data and machine learning technologies