Open-source AI: why it matters and how to get started
summary
Claims that open-source LLMs already match proprietary models and that almost 1 million models live on Hugging Face Hub. Concrete tooling and numbers: model selection via OpenLLM leaderboard and LLM Arena ELO votes, serving with TGI or llama.cpp or vLLM, and quantizing with 4- and 8‑bit quantization (bitsandbytes/GPTQ/AWQ). Fine-tuning options include TRL and LoRA where one can train 0.77% of parameters and deploy a quantized model with a single-line TGI start command.
AI-written summary · about
chapters (14)
- 0:00Open-source AI intro
- 3:18Why open-source models matter
- 5:05Benefits of open-source AI
- 7:33Workflow for open-source models
- 9:22Finding the right model
- 13:05Serving models tools
- 16:46Fine-tuning with Transformers
- 21:27LoRA fine-tuning
- 23:45Q&A: Fine-tuning
- 25:52Q&A: ELO Arena
- 27:16Q&A: Model benchmarks
- 29:21Q&A: Local LLM execution
- 31:11Q&A: Causal LM
- 33:06Q&A: Reasoning models o1
related talks
42mOpen Source LLMs — Powering AI Accessibility and Innovation
20mSelf-hosting AI LLMs - a beginners guide
29mLeveraging open-source LLMs for production
01h 02mA hacker's guide to open source LLMs
01h 49mAI engineering with open access LLMs that lie, curse, and steal
14mKeynote: Building AI with Open Source & Hugging Face
47mOwn Your LLM: Running Generative and Agentic AI Locally
36mInsights from The State of AI Report 2023