RAG (Retrieval Augmented Generation) with Mobile Application
summary
Builds a 100% native Android (Jetpack Compose) RAG app that uses OpenAI/Gemini APIs and Orama embeddings with Chroma on the server and migrates retrieval and generation on device using ONNX runtime and objectbox on-device vector DB. Shows quantized 4‑bit models ≈4GB as feasible versus ~60GB originals, reports LangChain-for-J reduces code by ~48%, and observes on-device inference works but is slow on midrange phones (needs ~10+ tokens/sec for good UX), so migration succeeds with clear performance and model‑size tradeoffs.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 1:10Artificial Intelligence
- 4:21Fine Tuning
- 5:15quation
- 6:55what is RAG
- 7:20why RAG
- 8:29RAG Example
- 8:48Development Process
- 10:57Embedding
- 12:25Implementation
- 18:23Process
- 26:23Security and Privacy
- 34:41Language Model
- 36:34Conclusion
related talks
14mGemma on mobile and web. Best and worst practices
20mRetrieval Augmented Generation for AI Apps
21mHow RAG is Redefining AI Capabilities
50mRetrieval Augmented Generation Beyond Toy Demos
53mBuild RAG from Scratch
27mFrom RAGs to riches: Build an AI document interrogation app in 30 mins
48mCrafting your own RAG system: Leveraging 30+ LLMs for enhanced performance
41mFrom RAGs to riches: Build an AI document interrogation app in 30 mins