Llama V2 in Azure AI for Finetuning, Evaluation and Deployment from the Model Catalog
summary
Llama 2 models (pretrained on about two trillion tokens) appear in the Azure ML model catalog as ready-to-run artifacts with 7B, 13B, 70B sizes and fine-tuned chat variants using over a million human annotations. The platform runs evaluation against your own test set, keeps training data inside the workspace, and supports fine-tuning with accelerators DeepSpeed, ORT, LoRA to reduce time and memory. Deployments enable online or batch endpoints with Azure AI Content Safety checking inputs and outputs; a demo fine-tune for dialogue summarization completes in hours and returns concise captions.
AI-written summary · about
chapters (10)
- 0:00Llama 2 on Azure AI
- 2:27Azure AI Model Catalog
- 3:50Fine-tuning and evaluation
- 6:00Content safety in deployments
- 8:26Demo: Model catalog
- 13:27Fine-tuning Llama 2
- 17:15Reviewing fine-tuning results
- 18:18Deploying fine-tuned models
- 19:45Using models in apps
- 20:47Q&A
related talks
48mPracticalities of Fine-Tuning Llama 2 with AI Studio
47mAccelerate your AI journey with Azure AI model catalog
53mLLM Quality Optimization Bootcamp
17mUnpacking the Llama Stack: Architecting Next-Gen AI Applications
31mBuilding LLaMA: Meta's Director of GenAI
51mTaking Models to the Next Level with Azure Machine Learning
30mNVIDIA AI Enterprise Registry for AzureML deployments of AI Workflows
25mFine-Tuning Large Language Models With Declarative ML Orchestration