News Crawler via Langchain.RB and Gemini APIs
summary
Builds a news crawler that crawls ~10,000 articles into a Rails app, embeds them and runs vector similarity via PGVector, then updates a base LLM using Gemini for multimodal extraction plus RAG and langchainrb gem tooling. Shows four demos: embedding nearest-neighbor search, prompt assembly with retrieved excerpts, calling a News API tool, and function-calling to save articles. Live system mostly works but demos fail about 30% from network, API limits, or parsing quirks.
AI-written summary · about
chapters (11)
- 0:00News Crawler project intro
- 1:02Speaker and Ruby advocacy
- 7:25Vector DB challenges in semantic search
- 10:13LLM limits with RAG and function calling
- 14:29Ruby News Crawler architecture
- 17:36Demo: Semantic article similarity
- 20:27Demo: RAG with prompt injection
- 24:17Demo: LLM tool calling with News API
- 27:51Demo: Interactive assistant
- 35:13Q&A
- 39:18Closing remarks
related talks
01h 04mBuilding LLM powered applications in Ruby
53mBuild RAG from Scratch
02h 47mFrom naive to advanced RAG: the complete guide
48mGoing Beyond Prompt Engineering
35mBuilding a Local News RAG: The Quest for Trustworthiness
22mBuilding Cypher Queries With Open-Source LLMs
01h 17mGraph Database and Retrieval Augmented Generation
01h 09mFlash Talks