Streamlining Metadata Management with PostgreSQL and Apache Gravitino
summary
Centralized metadata layer that federates existing meta stores (including PostgreSQL via a JDBC connector) and enforces pushdown policies at the query engine, with REST, Java and Python APIs plus a CLI and UI. Supports Iceberg, Paimon/Hoodie, file sets (FUSE written in Rust), model and topic catalogs, credential vending and Apache Ranger–backed fine‑grained access. Claims include 1 million downloads in nine months, active community with 1.3K GitHub stars, and production features like JDBC connector and FUSE in Rust.
AI-written summary · about
chapters (12)
- 0:00Metadata management challenges
- 3:50Data consumption and PostgreSQL
- 7:43What is a data catalog
- 10:54Key features of a data catalog
- 14:43Apache Gravitino for unified catalog
- 15:04Gravitino goals and architecture
- 18:57Federated metadata governance
- 21:51Live demo: Gravitino UI
- 25:19AI/ML with Gravitino metadata
- 28:18Gravitino interfaces and roadmap
- 31:54Community growth and open-source
- 32:15Q&A
related talks
31mApache Gravitino: A Multi-regional, Geo-distributed Meta Datalake
24mMetadata driven framework for data discovery and data quality
46mJourney of metadata
37mBreakdown New Data Silos in Generative AI
38mHerding Cat(alogue)s: The Apache Iceberg™ Catalog Landscape
01h 12mFrom Storage Formats to Open Governance: The Evolution to Apache Polaris
39mMetadata as a Control Plane: Inside the Apache Iceberg REST Catalog
31mHow Apache Iceberg Enables Multi-Engine Data Platforms