Why Dolly is just the beginning for open LLM models
summary
Shows that you can get usable open LLMs with tiny, focused datasets: Dolly 1.0 proves fine‑tuning a 2‑year‑old base model on 15,000 Q&A pairs yields solid results, Dolly 2.0 releases that Dolly 15K dataset and an instruction‑tuned model trained for ~$100; contrasts outcomes from Pythia, Llama and GPT‑4 and cites a concrete failure where Llama hallucinated proprietor of a bagel shop and a real leak where corporate prompts exposed Samsung secrets; demos run via nat.dev comparisons and Hugging Face models.
AI-written summary · about
chapters (8)
- 0:00Introduction to LLMs
- 1:26Risks of proprietary models
- 3:33Demonstrating hallucinations
- 7:39Data privacy concerns
- 10:53The Dolly proof of concept
- 13:33Data quality and performance
- 18:05Small models and ensembling
- 20:01Open source and conclusion
related talks
47mRamaLama: Making working with AI Models Boring
47mLaptop-sized ML for Text, with Open Source
53mLLM Quality Optimization Bootcamp
35mOpen-source AI: why it matters and how to get started
01h 35mSave your API Keys for someone else
18mSmolLM: The Rise of Smol Models
29mThe Evolution of LLMs: From Imitation to Reasoning
43mLarge Language Models for Devs: from zero to your first LLM application