Deep research at scale
summary
Blueprint for scaling an LLM pipeline to 15M+ documents with measurable results: use HPC-based retrieval plus hybrid search (BM25 + HNSW ANN), enrich documents with NER/entity normalization, chunk strategically, and index into a vector DB; evolve from LangGraph ReAct to DeepAgents with sub-agents, skills for progressive disclosure, reranking and LLM-as-judge, and MLflow/Bedrock for observability; achieve ~10% accuracy lift from enrichment, add model fallbacks, guardrails, rubric-driven CI/CD and human verification for production reliability.
AI-written summary · about
chapters (13)
- 0:00Introduction and overview
- 1:30Biomedical domain challenges
- 3:55Data pipeline and infrastructure
- 5:40Content enrichment strategies
- 7:37Scalable HPC processing
- 9:52Retrieval engine architecture
- 11:02Evolving agentic architectures
- 14:07Progressive disclosure with skills
- 15:22Evaluation frameworks
- 20:07Rubric taxonomy and axes
- 23:25Benchmarking prompt complexity
- 25:19CI/CD and quality gates
- 27:26Battle scars and takeaways
related talks
36mTechniques to avoid common LLM pitfalls
51mQuery Understanding With Large Language Models: Techniques and...
57mLLMs gone wild
23mSynthetic data that meets enterprise compliance needs
40mTechniques to avoid common LLM pitfalls
44mProduction-Grade LLMs & AI Agents
12mDeploying AI Literature Agents at Scale
01h 00mLLMs gone wild