Metadata as a Control Plane: Inside the Apache Iceberg REST Catalog
summary
Claims Iceberg’s REST catalog becomes the control plane for compute independence, governance and automation; a concrete implementation called Lakekeeper (Rust) REST catalog exists and demo proves clients (Trino, PyIceberg) can use short‑lived vended credentials so engines read object storage without static keys. Authorization uses OpenFGA + OPA bridge for centralized policies, the catalog exposes change events and supports branching/merge and multi‑table transactions. Adoption notes: Cloudflare R2 and EnterpriseDB mentioned and GitHub/docker metrics hit ~90,000 downloads/month.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 1:07Apache Iceberg for lakehouses
- 3:14From warehouses to lakehouses
- 8:31Iceberg REST catalog specs
- 11:25REST catalog governance
- 14:14Lakehouse as control plane
- 16:18LakeKeeper overview
- 17:01OpenFGA access governance
- 20:16Live demo: LakeKeeper access
- 24:32Control plane use cases
- 27:25Data contracts governance
- 30:13AI governance outlook
- 31:36Key takeaways
- 34:50Future of data products
- 35:11Q&A
related talks
33mZero Trust with Iceberg REST Catalog and Policy Engines
31mHow Apache Iceberg Enables Multi-Engine Data Platforms
37mApache Iceberg: An Architectural Look Under the Covers
34mApache Iceberg.
52mApache Iceberg - An Architectural Look Under the Covers
41mTip of the Iceberg
01h 12mHands-on with Apache Iceberg
46mTake the Polar Plunge: A Fearless Introduction to Apache Iceberg™