Simplifying the adoption of generative AI for enterprises
summary
NVIDIA demonstrates an end-to-end enterprise path: train large LLMs on H100/P5 or A100/P4 with Megatron Core and NeMo, curate trillions of tokens with GPU-accelerated deduplication, apply FP8/FP16 Transformer Engine speedups, then deploy with Triton + TensorRT-LLM to get 3x lower latency and 5x throughput (catalog/search examples) and sub-50 ms responses for high-throughput search. NGC containers, SageMaker integration, Jetson edge deploys, and an auto-configurator for TP/PP sizing and hyperparameter sweeps round out the stack.
AI-written summary · about
chapters (10)
- 0:00Introduction to generative AI adoption
- 1:48NVIDIA AWS collaboration overview
- 8:01H100 GPU and transformer engine
- 11:31NeMo framework for generative AI
- 22:25Challenges in LLM training
- 30:49Triton Inference Server introduction
- 36:32Triton optimizations and model analyzer
- 42:56TensorRT for model optimization
- 45:48TensorRT-LLM for large models
- 51:14Call to action and conclusions
related talks
28mHarnessing generative AI with NVIDIA AI and Microsoft Azure
28mScaling agentic AI: From infrastructure to enterprise outcomes
54mDeep learning on AWS with NVIDIA: From training to deployment
42mFrom theory to practice: The rise of Generative AI
53mHow to Build and Scale Ultra-Lean Gen AI Startups
24mScalable GenAI in Production
29mBuild Customize and Deploy LLMs At-Scale on Azure with NVIDIA NeMo- 28mScaling agentic AI: From infrastructure to enterprise outcomes – Kevin Cochrane