How to Build a Modern Data Lakehouse?
summary
A practical recipe for building a cloud Data Lakehouse on Databricks using Delta Lake, Spark and object storage (Parquet on S3/Azure/GCS). Shows migrating from an expensive data warehouse to a single lakehouse that reduces compute costs by shutting down clusters, speeds queries, and provides ACID transactions, schema evolution and time travel while combining batch and streaming ingestion. Demonstrates SQL, streaming ingestion, and versioned updates for analytics and machine learning with a live demo of table growth.
AI-written summary · about
chapters (13)
- 0:00Wprowadzenie i cel prelekcji
- 2:10Dane jako fundament biznesu
- 6:02Tradycyjne metody przechowywania danych
- 9:16Hurtownia danych i jej ograniczenia
- 11:47Data Lake – elastyczność i wady
- 16:43Problem Data Swamp w Data Lake
- 19:10Koncepcja Data Lakehouse
- 24:36Architektura i działanie Lakehouse
- 28:57Demo: Budowa Lakehouse w Databricks
- 36:08Przypadek wdrożenia u klientów
- 43:35Kluczowe funkcjonalności Delta Lake
- 48:33Podsumowanie i zalecenia
- 49:16Dlaczego Data Lakehouse to przyszłość?
related talks
21mWhy the Lakehouse?
28mLakehouse Architecture with Delta Lake and Databricks
20mDelta Lake: Turn your Data Lake into a Lakehouse
34mData Lakehouse: Who, Why and How
40mData Lake House in the Cloud: The Azure Solutions
26mModernize your Enterprise Data Warehouse & generate value from data
58mBuild reliable data lakes with Delta Lake & Databricks
44mWhat's the Best Big Data Architecture for You?