No Cloud- No Problem. Local RAG with Embedding Gemma
summary
Claim: full RAG and topic classification run locally using Embedding Gemma (308M parameters) with Sentence Transformers and Hugging Face, storing vectors in FAISS. Implementation chunks a Q3 Alphabet earnings PDF into 64‑word pieces (≈160 chunks), trains a logistic‑regression classifier and measures recall@5; Embedding Gemma (768‑dim) slightly outperforms a 1024‑dim Quen and 384‑dim all‑mini by capturing specific queries like YouTube shots monetization. Results validate low‑cost on‑device retrieval.
AI-written summary · about
chapters (7)
- 0:00Local RAG with Embedding Gemma
- 1:13Word embeddings basics
- 4:01Embedding Gemma explained
- 8:42Embedding Gemma performance
- 12:42Gemma vs other models
- 18:45Real-world performance test
- 20:54Q&A
related talks
56mRAG: from dumb implementation to serious results
53mBuild RAG from Scratch
30mRAG: Tricks from the trenches.
49mRAG: from dumb implementation to serious results
14mGemma on mobile and web. Best and worst practices
48mCrafting your own RAG system: Leveraging 30+ LLMs for enhanced performance
10mDemo: Using Gemma with the Hugging Face ecosystem
19mRAG in 2025: State of the Art and the Road Forward