Keynote: Build Accelerated LLM Applications with Open Source Tools from NVIDIA
Talkat Cassandra Summit 2023 · on Wednesday, 2023-12-13· at 09:35· Grand Ballroom BC (Concourse Exhibit Level)·
summary
NVIDIA claims 500 open-source projects and says over 100 million RTX PCs exist to run them, then demos concrete tools: RAPIDS accelerator runs existing pandas scripts unmodified on GPU, Megatron-LM scales Transformer training across thousands of GPUs, MRAW is a sparse mixture of experts 47B→13B per token for efficiency, Triton serves models on GPU/CPU with many backends, RAFT provides GPU-accelerated vector search, and NeMo Guardrails enforces safety rules in deployed LLM apps.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 0:47Nvidia has over 500 open source projects
- 1:30The LLM Stack
- 2:34Open Source Tools
- 2:56AI Foundations Models
- 4:06Data Processing
- 5:18Rapid
- 5:37Spark
- 5:51Megatron LM
- 7:07Nemo Toolkit
- 7:41Triton
- 8:21Backends
- 8:55Rag
- 9:26Raft
- 10:46Rails
- 11:15Outro
related talks
12mKeynote: Build Accelerated LLM Applications with Open Source Tools from NVIDIA
24mKeynote: Next-generation frameworks for Large-scale AI
13mKeynote: To Nemotron and Beyond: Building the Open Source AI Ecosystem
12mKeynote: How LLMs Have Changed the Way Organizations Do AI
52mSimplifying the adoption of generative AI for enterprises
18mKeynote
32mKeynote: AI & the stuff built for AI - are they actually useful for data science?
08mKeynote: Plug in and Scale: Serving LLM Models on Kubernetes Ma...