Bypassing the Multimodal Tax: Hybrid RAG, SQL RRF & UI Telemetry
summary
Claims a working local-first RAG prototype that cuts cloud vision token bills by converting docs to Markdown and running OCR locally with local OCR pipelines on corrupted pages. It uses Postgres + pgvector with raw SQL, HNSW indexes and a hybrid dense+BΜ25 retrieval mixing scores via Reciprocal Rank Fusion (RRF), returns configurable top‑k (demo defaults to top 2), and shows concrete chunking parameters like 512-character chunks with 64% overlap. Stack: Python/FastAPI, React, Docker, Ollama/Qwen 2.5, LangFuse.
AI-written summary · about
chapters (17)
- 0:00Introduction
- 1:08Document upload problems
- 2:56Talk overview
- 4:21Structured document processing
- 9:21Chunking strategies for RAG
- 16:16Demo: heading-based chunking
- 20:30Paragraph and fixed-length chunking
- 22:00Sentence and image chunking
- 26:17Python agents for speed
- 29:08Hybrid search with BM25
- 33:03Reranking and retrieval
- 35:31Telemetry with LangFuse
- 37:36Guardrails implementation
- 40:40Testing guardrails
- 41:20Widget integration
- 42:25Model selection
- 44:35Key takeaways
related talks
35mThe Levels of RAG 🦜
31mOrchestrating Multimodal RAG With Agentic Workflows
30mI Want to Build a RAG System, Now What?
27mOptimizing RAG Systems in Practice: Challenges and Proven Solutions
59mRAG Against the Machine
25mAdvanced RAG Systems: Scaling Retrieval-Augmented Generation in Production
26mAdvanced Document Parsing for AI Workflows
40mHow [not] to evaluate your RAG