Socialising the Elephant with the Rest of the Animals - Integrating...
summary
Egeria provides an open, peer-to-peer metadata layer that maps vendor schemas to a common type system and runs in production at IBM/Atruvia to give a single, federated view across warehouses, lakes and meshes. Implementations include an Apache Atlas connector, a Hive Metastore connector and lineage ingestion via Open Lineage; connectors publish into a cohort so repositories exchange metadata rather than centralize it. The project claims automatic enrichment, stitchable lineage and live governance automation and shows working deployments.
AI-written summary · about
chapters (8)
- 0:00Introduction and context
- 2:13Evolution of data management
- 8:02Architecture challenges
- 13:25Modern architectural patterns
- 21:16Introduction to Egeria
- 27:46Technical implementation
- 36:21Big data integration
- 39:56Q&A and conclusion
related talks
40mBecome an data driven organization through unified metadata using ODPi Egeria
45mTechnical Challenges in Open Metadata
46mJourney of metadata
33mStreamlining Metadata Management with PostgreSQL and Apache Gravitino
11mIntroducing an Enhanced Metadata Broker for Manufacturing Data Spaces
37mMetaOps: Metadata Operations For End-To-End Data & Machine Learning Platforms
38mHerding Cat(alogue)s: The Apache Iceberg™ Catalog Landscape
16mEvolution of Extract-Transform-Load (ETL) processes towards data product pipelines