Practicalities of Fine-Tuning Llama 2 with AI Studio
summary
Step-by-step recipe for turning domain PDFs into a reliable RAG model: split documents into chunks, generate ~18,000 QA pairs (using a Llama 3 70B instruct model) with Chain-of-Thought and distractor chunks, bundle into train/val/test, then fine-tune a base Llama 7B or Llama 27B in Azure AI Studio using batch=1, lr≈0.002, 1 epoch. Dataset generation consumes ~10M input / 3.7M output tokens (~$80), fine-tuning costs ~$35/hr (≈1–2 hrs), and yields a 4.76 similarity score (~1.7× improvement) while improving grounded retrieval behavior.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 1:05Retrieval Augmented Gen
- 5:50Fine-Tuning Overview
- 10:12What is Raft?
- 17:26Data Preparation
- 23:38Fine-Tuning in AI Studio
- 31:21Model Deployment
- 32:47Raft Demo
- 37:18Cost and Results
- 43:58Q&A and Conclusion
related talks
09mTailoring Llama for Your Needs: Fine-Tuning & Evaluation
29mLocal LLM fine-tuning: a practical example
23mLlama V2 in Azure AI for Finetuning, Evaluation and Deployment from the Model Catalog
53mLLM Quality Optimization Bootcamp
48mDeploy large language models responsibly with Azure AI
30mLearning to wield large language models
21mFrom Data to Precision: Fine-Tuning LLMs for Proprietary Query Languages
44mPimp My Llama - How to Customize LLMs for your Business Case