Applying Open Source Methods to Building and Training Large Langu...
summary
Project creates a workflow that lets contributors add 5-example YAML skills or document-sourced knowledge to an upstream model without full fork or fine-tune, uses a taxonomy-driven curation step, large-scale synthetic generation (five examples -> thousands), and phased alignment tuning, exposes an iLab CLI to run quantized local models (Mac Metal GPU demo) and a backend that validates, evaluates and produces base model vN+1 releases; cites an ecosystem problem: 6000 forks in three weeks after Llama 3 and backend openness is still evolving.
AI-written summary · about
chapters (13)
- 0:00Open source AI methods intro
- 3:15Open AI models state
- 6:33Open model contribution challenges
- 9:04InstructLab methodology
- 11:13Taxonomy and synthetic data
- 13:21Skills vs knowledge in models
- 16:11Contributing skills and knowledge
- 19:39Local InstructLab experiments
- 21:27InstructLab CLI demo
- 23:56Backend training and evaluation
- 25:01Open source LLM workflow
- 26:46Get involved in InstructLab
- 29:38Q&A
related talks
40mInstructLab: Applying Open Source Methods to Building and Trai...- 40mInstructLab: Applying Open Source Methods to Building and Trai...
28mA community based open source LLM and what it can do for your business
42mFast & Furious: From Zero to Open Source Community in 9 Weeks
44mOpen-Source and Science in the Era of Foundation Models
31mThe AI Revolution Will Not Be Monopolized: How open-source beats economies of scale, even for LLMs
43mLarge Language Models for Devs: from zero to your first LLM application
01h 49mAI engineering with open access LLMs that lie, curse, and steal