From Pretrained To Personal: Privacy-First...
summary
Claim: local AI PCs now run meaningful fine-tuning so sensitive data never leaves the device. Demonstrated setup uses PyTorch 2.10 + ROCm 7.12 on an AMD Strix Halo (128 GB RAM, 120 W) and compares LoRA and QLoRA versus full fine-tuning. Experiments fine-tune CoRe 3.5-family models from 0.8B up to 27B fine-tune (100 optimizer steps) showing a 27B model trains on-device in hours/overnight with small datasets; tooling options include Llama Factory and Unsloth and an upcoming AMD Playbooks guide.
AI-written summary · about
related talks
12mMemory-Efficient AI: How PEFT and PyTorch Enable Accessible LLM Fine-Tuning
21mBeat GPT 4 and Build the Best AI For You: Mobilizing First-Party Data with Fine-Tuning
28m🧼 From GPU-poor to data-rich: data quality practices for LLM fine-tuning
21mBeat GPT 4 and Build the Best AI For You: Mobilizing First-Party Data with Fine-Tuning
33mBuild Better AI with First-Party Data and Fine-Tuning
42mPost-Training Small Language Models: the adventures of a practitioner
22mTraining a LLaMA in your Backyard: Fine-tuning Very Large...
16mtorchtune: Easy and Accessible Finetuning in Native PyTorch