One Configuration, Many Imports
summary
A uniform configuration called import-spec maps Parquet sources to Neo4j nodes and relationships and generates Neo4j Admin headers and schema files automatically; back ends shown are Neo4j admin, Apache Beam and Spark, all reusing the same YAML spec. A plugin system adds custom source types, and a declarative Execution plan exposes groups and sequential stages so Spark jobs or Beam pipelines run in the correct order. Demo imports run tests and the generated graph removes the inventory indirection as intended.
AI-written summary · about
chapters (11)
- 0:00Welcome and introduction
- 1:07Current data import challenges
- 5:18Introducing import spec
- 7:45Demo data and setup
- 9:12Import spec configuration explained
- 13:55Neo4j admin backend example
- 16:47Plugin system for sources
- 18:32Apache beam backend example
- 20:18Spark backend with execution plan
- 24:06Recap of demo and benefits
- 27:19Closing and resources
related talks
30mOne Import Tool to Rule them All
31mNeo4j in the Cloud With NoCode Imports
42mNeo4j in the Cloud with NoCode Imports
36mData pipelines engineering made simple with Scala
21mSerious Network Analysis Using Hadoop & Neo4j
28mNeo4j for Beginners: Pros and Cons, Based on Social Networks
15mFaster Data Imports With Concurrent Transactional Subqueries
31mApache Gravitino: A Multi-regional, Geo-distributed Meta Datalake