RAG Pipeline Evaluation
summary
Built an evaluation pipeline that runs Neo4j-backed RAG pipelines against a dataset of 10,000 movies and 20,000 persons, compares three retriever strategies — pure vector search, augmented vector search with graph traversals, and text-to-Cypher — and scores outputs with the open-source RAGAS framework. Results show graph-augmented strategies and text-to-Cypher beat pure vector on context precision/recall and generation faithfulness, text-to-Cypher handling aggregations and long traversals; metrics are computed with GPT‑4o; codebase and Neo4j JNI tools are in beta.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:27Motivations for RAG
- 4:42Evaluating RAG challenges
- 6:08Introducing Ragas
- 9:23Retrieval strategies overview
- 14:45Demo setup with Neo4j
- 19:04Text-to-Cypher demo
- 22:22Results with Ragas metrics
- 24:05Strategy comparison results
- 28:41Conclusions and future work
- 30:27Q&A
related talks
23mMeasuring the Performance of VectorRAG vs GraphRAG
35mGraphRag in Action: Revolutionizing Fraud Detection Techniques
39mEnhancing RAG with Neo4j Knowledge Graph
01h 17mGraph Database and Retrieval Augmented Generation
16mEnhancing Retrieval-Augmented Generation with GraphRAG Patterns in Neo4j
27mAdaptive Knowledge Graphs for Dynamic RAG Systems
27mOptimizing RAG Systems in Practice: Challenges and Proven Solutions
30mPerformant GraphRAG Improving Neo4j to Drive Autonomous AI ...