Build your data lakehouse with Starburst Galaxy
summary
Starburst Galaxy connects S3 and MongoDB catalogs, uses Glue metastore, and runs a small cluster (execution mode off) to convert CSV spawn data and MongoDB Pokedex into Iceberg ORC tables, removing two unused columns and filtering to Bay Area lat/long. It creates land, structure and consume schemas, joins spawn with Pokedex to produce a read-only consume table, enforces table- and location-level privileges (consume role denied UPDATE), and populates a ThoughtSpot liveboard; query history shows the exact SQL used.
AI-written summary · about
chapters (8)
- 0:00Intro
- 0:23Data Lakes
- 1:43Why use Starburst
- 2:14How to choose
- 2:46OpenTable formats
- 3:25Native security
- 4:24Organization
- 6:38Demo
related talks
25mBuild open format data lakehouse
29mServing a multi-platform Data Lake via Delta, Iceberg, Unity and Glue
46mBuild an in-place Amazon S3 data lake with
59mModern data lakes with Lake Formation, Amazon Redshift & AWS Glue
18mOpen-Source OLAP on Open-Source Data Warehouses.
45mBuilding an Open Data Lakehouse using Apache Spark, Apache Iceberg and Dremio
27mBuilding a PostgreSQL data warehouse
25mHow to build a platform for AI and analytics based on Apache Iceberg