Build Customize and Deploy LLMs At-Scale on Azure with NVIDIA NeMo
summary
NVIDIA NeMo lets teams train, customize and deploy LLMs on Azure with concrete tooling: a Data Curator, auto-configuration for tensor/pipeline/sequence parallelism, and RLHF and parameter-efficient methods (LoRA, adapters, PEFT) for customization; Nemo Guardrails constrains dialogue and provides citation-backed retrieval; sequence-parallelism and selective recompute yield 29–31% memory improvement; scale tests reach up to 530B parameters on 1,400 GPUs, and a live demo runs a 5B GPT model on A100v4 nodes with InfiniBand achieving 37.5s per global step on a 5B GPT.
AI-written summary · about
chapters (8)
- 0:00Start
- 5:20NeMo Expanding Support Across Modalities NeMo offers multi-modality support
- 6:39Building Generative Al Foundation Models Efficiently and quickly training models using NeMo
- 9:09Customization Techniques for Generative Al Making models useful through SOTA techniques on NeMo
- 10:41Large-Scale Inference for Generative Al Deploying Generative Al models at-scale for production use-case with NeMo
- 11:17NeMo Guardrails for Large Language Models Add Boundaries Ensure Chatbots Operate According to Use Cases
- 12:53NeMo Framework Performance Performance on NVIDIA A100 DGX SuperPODs
- 19:03NeMo Framework Performance Performance on NVIDIA A100 Tensor Core GPUs on Azure
related talks
52mSimplifying the adoption of generative AI for enterprises
31mPanel: The Future of AI
23mNeMo: Open Source Toolkit for Conversational AI
54mDeep learning on AWS with NVIDIA: From training to deployment
12mKeynote: Build Accelerated LLM Applications with Open Source Tools from NVIDIA
22mPerformance Optimizations at 100K+ Scale- 12mKeynote: Build Accelerated LLM Applications with Open Source Tools from NVIDIA
50mAccelerate development and deployment of Gen AI with NVIDIA on Azure