Building & Deploying LLM Apps
summary
Claim: Llama 2 handles text summarization at cost roughly 30× cheaper than GPT‑4 while matching GPT‑3.5 Turbo on many tasks. Tools: Ray is used to parallelize per-file inference and cut latency, Anyscale Endpoints exposes Llama 7/13/70B, and retrieval‑augmented generation with a vector DB returns cited sources. Experiments: fine‑tuning shifts style but not entrenched facts (Romeo stays Romeo), and two apps are built and deployed — Documentor GitHub bot (comments PRs, works, scales) and an Ask AI doc chatbot.
AI-written summary · about
chapters (17)
- 0:00Introduction
- 1:38Large Language Models
- 5:33Rise of the AI Engineer
- 7:02OpenAI
- 8:10Ray
- 9:50What is Ray
- 12:11Example
- 13:25About Max
- 15:14Open Source Models
- 17:45When do you need which model
- 20:02Fine tuning
- 21:32Demos
- 21:54Documentor
- 23:00Documenter
- 27:18Ray Documentation
- 30:13QA
- 31:06Retrieval Playbook
related talks
42mUsing LlamaIndex for building Generative AI apps
09mTailoring Llama for Your Needs: Fine-Tuning & Evaluation
55mBuild enterprise generative AI apps using Llama 3 at 1,000 tokens/s on the SambaNova AI platform
37mExpanding the Capabilities of LLMs Through Community Collaboration
53mLLM Quality Optimization Bootcamp
22mThe Disruptive Potential of On-Device LLMs
27mProductionizing Open Source LLMs
27mAI Demystified Unveiling the Intelligence Behind the Machine