Making AI transparent with RAG on your own data
summary
Pilot uses RAG on a scanned book (Edgot Parti) to generate source-backed Wikipedia edits by vectorizing chunks, running NER with spaCy, storing vectors in MariaDB vector and querying embeddings (OpenAI) to feed Claude/GPT. Process finds 278 people, 265 places and produces suggestions that the author rates one third accepted, one third modified, one third dismissed. Main failures are entity disambiguation and overconfident hallucinations; chunk size and context window tuning prove decisive.
AI-written summary · about
chapters (10)
- 0:00Transparent AI with RAG
- 3:28Project Fredrika challenge
- 9:39AI-assisted Wikipedia editing
- 17:06Technical implementation
- 23:59Open source RAG databases
- 27:56RAG technical steps
- 31:09Effective RAG prompts
- 34:19MariaDB for RAG scalability
- 36:06Adapting RAG use case
- 37:34Q&A chunkification & search
related talks
39mRAGs to Riches: How Our Content Affects Retrieval Augmented Generation
41mFrom RAGs to riches: Build an AI document interrogation app in 30 mins
53mBuild RAG from Scratch
22mThe Power of RAG at Scale: A Year of Explosive Growth and Observab...
44mBuild RAG from Scratch
27mOptimizing RAG Systems in Practice: Challenges and Proven Solutions
35mThe Hidden Jewels to Solve Confabulation Challenges in Generative AI’s RAG Promise
19mImproving RAG Systems - Key Challenges and Solutions