Data warehousing reinvented for today's needs
summary
A cloud pipeline shows how to build a scalable analytics DWH end to end: S3 as landing and historical store, AWS Glue crawlers and Glue Studio ETL, Redshift (serverless) for hot analytics, Redshift Spectrum for querying lake data, Redshift ML for forecasting and QuickSight for dashboards and natural‑language queries. The demo denormalizes orders/products, loads 22 million rows and completes the job in 10 minutes, uses serverless Redshift RPUs for on‑demand compute, trains an ML forecast and publishes working dashboards.
AI-written summary · about
chapters (20)
- 0:00Modern data warehousing intro
- 1:09Data volumes exploding
- 3:39Traditional warehouse problems
- 4:40Modern warehouse traits
- 5:43Amazon Redshift as modern DW
- 8:55Hot vs cold data storage
- 11:02ETL vs ELT and AWS Glue
- 14:37QuickSight for insights
- 15:20Demo setup overview
- 17:08Cataloging in S3
- 20:08AWS Glue crawler setup
- 22:56ETL jobs with Glue Studio
- 28:19Running ETL and loading data
- 29:04Redshift serverless console
- 31:33Querying Redshift data
- 32:56ML models with Redshift ML
- 35:06Historical data with Spectrum
- 36:53Dashboards in QuickSight
- 39:42Natural language queries
- 40:25Q&A
related talks
59mModern data lakes with Lake Formation, Amazon Redshift & AWS Glue
01h 02mData warehousing reinvented for today’s needs
28mTurn Data Into Insights
48mData warehousing reinvented for today’s needs
30mData Engineering building your BI infrastructure from scratch
24mModernizing the Data Warehouse
47mConstruyendo Lagos de Datos y Analíticos en AWS + Datos Abiertos
56mUnderstanding and achieving a modern data architecture