Exploring Large Graphs at the Heart of the French National Audiovisual Ins.
summary
French INA metadata model uses a cultural‑heritage graph and powers ~60 million documents stored as relational rows, MongoDB fragments and Elastic indexes; indexing time falls from ~3 weeks to less than 48 hours by precomputing JSON fragments; transformations use gslt to flatten into Lucene‑friendly JSON; deployment runs on‑prem Elasticsearch with microservices on Kubernetes, message queues for async jobs and Talent for orchestration; search latency averages under 200 ms and ongoing work adds embeddings and relevance tuning.
AI-written summary · about
chapters (10)
- 0:00Project introduction
- 3:03Data lake architecture
- 8:27Large graph modeling
- 15:54Indexing with JSON fragments
- 21:13Data transformation with GSLT
- 24:03Indexing workflow
- 28:28Search API design
- 35:00Performance evaluation
- 37:10Key takeaways
- 39:17Q&A
related talks
44mSearching through large graphs using Elasticsearch
45mBehind the hype: managing billion-scale embeddings in Elasticsearch and OpenSearch
49mModern Graph Databases
22mHas This Happened Before? Queryable Memory for Database Logs
30mPerformant GraphRAG Improving Neo4j to Drive Autonomous AI ...
28mCreating Enterprise Knowledge Graphs from Unstructured Data
01h 00mGraph Mastery to AI
47mMake sense of your (big) data using Elasticsearch