Build Better AI with First-Party Data and Fine-Tuning
summary
Claim that small fine-tuned models on first‑party data beat giant LLMs on domain tasks with concrete numbers: Fashion MNIST examples show GPT‑4 mini ~81% vs XGBoost 90% vs ResNet 95% accuracy, legal retrieval with generic embeddings drops to ~49% vs web retrieval ~95%, and Databricks trains a tiny model that matches larger model accuracy at roughly twice the speed. Uses PyTorch and Hugging Face trainer for training, runs distributed jobs on Kubernetes via Kubetorch from Runhouse, reducing iteration and multi‑GPU deployment friction and enabling reproducible RL and fine‑tuning loops.
AI-written summary · about
chapters (16)
- 0:00Introduction and motivation
- 1:14Why fine-tune models
- 4:03Marketing myths about training
- 5:04Pre-training vs post-training
- 7:12Collecting training data
- 9:19Prompt engineering limitations
- 11:25ML infrastructure challenges
- 13:30Development and deployment pain
- 14:54Experiment escape velocity
- 15:37Snowflake analogy for ML
- 16:39Kubernetes for ML workloads
- 18:23CubeTorch solution overview
- 20:28Practical fine-tuning approaches
- 21:35Building high-quality datasets
- 23:18Key takeaways and next steps
- 25:43Q&A session
related talks
21mBeat GPT 4 and Build the Best AI For You: Mobilizing First-Party Data with Fine-Tuning
21mBeat GPT 4 and Build the Best AI For You: Mobilizing First-Party Data with Fine-Tuning
19mBeat GPT 4 & Build the Best AI For You
28mFine-Tuning Small Language Models with Python
02h 50mUnderstanding and Using AI Language Models
39mCollect Data to Fine-Tune Your Large Language Model
28m🧼 From GPU-poor to data-rich: data quality practices for LLM fine-tuning
10mFrom Pretrained To Personal: Privacy-First...