Context is King
summary
Argues whether Retrieval-Augmented Generation stays necessary versus dumping documents into huge windows, and shows concrete numbers: ~100k tokens is a practical cliff, 100k→1M costs 4x–10x latency, and a case where context raises accuracy from ~60% to 85%. Compares providers (Gemini fast, Claude cautious, GPT5 variable), tests distractors, reranking, and real RAG pipelines on Wikipedia; reranking improves retrieval metrics but often not final answer quality. Includes engineering notes, token-costs, and when long context vs RAG actually works.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 1:11Long context models challenge RAG
- 3:19LLM speed and performance research
- 5:46Promise and limits of long context
- 7:08Agentic coding with long context
- 12:24Latency scales with context size
- 17:45Model speed comparison
- 18:06Needle in a haystack benchmark
- 19:30Context rot research findings
- 24:26Distractors impact on models
- 24:48Long Q&A performance results
- 27:42Reranking with long context models
- 34:25RAG vs context conclusions
- 38:18Monday morning steps
- 39:01Summary
- 39:45Q&A
related talks
01h 00mThe Frontier between Retrieval-augmented and Long-context Language Models
56mRAG: from dumb implementation to serious results
25mAdvanced RAG Systems: Scaling Retrieval-Augmented Generation in Production
28mOptimizing Context Tuning for RAG Systems
12mUnconference: Context is all you need?
50mRetrieval Augmented Generation Beyond Toy Demos
49mRAG: from dumb implementation to serious results
19mRAG in 2025: State of the Art and the Road Forward