AI engineering with open access LLMs that lie, curse, and steal
summary
Shows concrete fixes for open access LLM failures: uses Llama 3 and a 2B Gemma demo, builds a RAG pipeline with 700-char chunks, embeddings and LanceDB to pull exact Linux kernel docs, then injects context to stop hallucinations. Demonstrates model-graded checks (LLM critic scoring ~7.5/10) and toxicity/pii filters, explains why 8–13B quantized models suffice for many apps and why 95% of use cases don’t need fine-tuning. Introduces control vectors to flip behavior without retraining.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:45Generative AI models overview
- 10:41Open vs proprietary LLMs
- 21:41Demo setup and access
- 32:50Prompting strategies
- 49:29Retrieval augmented generation
- 1:06:30Dealing with lies and hallucinations in LLMs
- 1:18:57Cursing and responses
- 1:31:44Control vectors
- 1:38:57Security risks
- 1:42:11Closing and Q&A
related talks
42mOpen Source LLMs — Powering AI Accessibility and Innovation
02h 52mLow Level Technicals of LLMs
57mLLMs gone wild
15mOpenAssistant's Opensource LLMs Saga
35mOpen-source AI: why it matters and how to get started
33mOpening Keynote: LLMs, between fantasy and reality - the poin...
28mProduction-Ready LLMs on Kubernetes: Patterns, Pitfalls, and Performa...
40mTechniques to avoid common LLM pitfalls