Workflows of Highly Functional App & Data Engineering Teams
summary
Describes a practical enterprise stack: use bronze/silver/gold medallion for raw → cleansed → curated data (bronze alone already beats ~90% of companies; gold is rare), keep schema in source control with database projects that build, diff and publish to containers, unit test stored procedures and run analyzers in CI/CD, avoid ad hoc Databricks notebooks for orchestration, and replace CRUD server code with Data API Builder to cut custom API surface to about 10% while retaining security and versioned schema management.
AI-written summary · about
chapters (6)
- 0:00Data engineering workflows
- 6:35Data origins in enterprises
- 13:56Medallion strategy
- 24:49Ad-hoc ETL pitfalls
- 33:37Developer tools for data teams
- 48:16Why data API layers
related talks
01h 05mWorkflows of Highly Functional App & Data Engineering Teams
41mModern Data Stack: Optimising and Scaling Data in a Tech Company
56mData Unleashed: A Developer's Perspective on Navigating the Architecture Maze
47mHow to pick the best Data transformation strategy
44mAgile Data Warehousing/Business Intelligence: Addressing the Hard Problems
19mDemystifying Modern Data Pipeline Architecture
27mHow To Develop & Operate An Agile Cloud Datawarehouse In A Rapidly Changing Multi-ERP Environment
31mData Unleashed: A Developer's Perspective on Navigating the Architecture Maze