Anatomy of an LLM
summary
Anatomy of an LLM gives concrete tactics: build a RAG pipeline with a vector DB (MongoDB Atlas Vector Search cited), use LangChain/LlamaIndex/DSPY and Hugging Face for corpora, and treat observability as evaluations for correctness, relevance and hallucination management. Chunking and embedding choice matter—chunking raises cosine similarity from ~57% to ~63%, and certain embeddings push it to 70–80%. A command-history RAG prototype returns previous shell commands and surfaces clusters; nomic visualization even reveals unexpected links, exposing both insights and security risks like poisoned embeddings.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 4:09Generative AI education urgency
- 9:07Neural networks and transformers
- 12:13Embeddings and vectors
- 14:19CIO’s shadow AI dilemma
- 23:29Industrial RAG systems
- 26:05Data engineering for answers
- 33:28Evaluating LLM outputs
- 36:40Cost of technical debt
- 41:17Shadow AI knowledge worker problem
- 44:04LLM security risks
- 49:02Legacy systems for AI
- 52:11Q&A
related talks
44mGive Your LLMs a Left Brain
48mCrafting your own RAG system: Leveraging 30+ LLMs for enhanced performance
47mGive Your LLMs a Left Brain
52mGive Your LLMs a Left Brain
57mLLMs gone wild
01h 33mRAG How To Talk To Your Data
01h 19mBuilding an LLM on your Laptop to...
07mLightning Talk: Grounding LLM to Avoid Hallucinations Using Vector S...