Fine-Tuning Large Language Models with Declarative ML Orchestration
summary
Claims that declarative ML orchestration makes LLM fine-tuning practical for smaller teams by assigning resources per task and cutting cloud waste, demonstrated by a Colab/Union AI workflow that fine-tunes a BT model from Hugging Face and evaluates it successfully. Uses Flyte on Kubernetes plus DeepSpeed optimizations, quantization and LoRA/QLoRA techniques, with workflows that set CPU/GPU per task. Notes Flyte has 6,000 GitHub stars and cites an example GPU cost of roughly $40/hour.
AI-written summary · about
chapters (15)
- 0:00Welcome
- 1:23Why fine-tuning LLMs matters
- 3:31Fine-tuning basics
- 8:04Fine-tuning techniques
- 11:11Supervised fine-tuning
- 13:38Model parallelism in training
- 15:02Optimizing with ZeRO and quantization
- 17:09LoRA for efficient fine-tuning
- 19:17Cost challenges in fine-tuning
- 19:38Declarative ML orchestration
- 21:23Flyte for ML orchestration
- 22:27Flyte with Kubernetes
- 24:11Cost control infrastructure
- 26:22Demo: Supervised fine-tuning
- 30:21Summary
related talks
25mFine-Tuning Large Language Models With Declarative ML Orchestration
39mThe Future is Fine-Tuned: Deploying Task-specific LLMs
16mStreamline LLM Fine-tuning on Kubernetes With Kubeflow LLM Trainer
30mLearning to wield large language models
31mDeploying Large Language Models with Ease
21mFrom Data to Precision: Fine-Tuning LLMs for Proprietary Query Languages
21mStreamline LLM Fine-Tuning on Kubernetes with Kubeflow LLM Trainer
43mCreating Your Own Copy with a Fine-Tuned LLM