NeMo: Open Source Toolkit for Conversational AI
summary
NeMo is an open source conversational AI toolkit (released under Apache 2.0) that ships as pip or Docker packages and pretrained .nemo archives on NGC and Hugging Face. It trains with PyTorch Lightning and Megatron Core to scale to 1 trillion parameters and thousands of GPUs, supports ASR/LLM/TTS and upcoming multimodal code, and offers online streaming ASR that transcribes 160 ms chunks with WER and real‑time factor reporting. Demos include a working streaming ASR pipeline, a Gradio VAD+keyword “snake” UI with some latency, and TTS speaker interpolation via mixed speaker embeddings.
AI-written summary · about
chapters (13)
- 0:00NeMo introduction
- 1:03Installation and models
- 2:47NeMo fundamentals
- 3:50NeMo training start
- 5:14Automatic Speech Recognition basics
- 8:04Streaming ASR demo
- 9:50Voice snake game demo
- 10:51Large Language Models in NeMo
- 13:15Scaling LLMs with Megatron
- 15:43Text-to-Speech methods and models
- 16:45TTS speaker interpolation demo
- 18:12Resources and closing
- 18:55Q&A
related talks
36mNVIDIA NeMo: Toolkit for Conversational AI
29mBuild Customize and Deploy LLMs At-Scale on Azure with NVIDIA NeMo
01h 30mFrom Synthetic Examples to Production Signals
16mThe End of Awkward AI Transcriptions
21mMultimodal Open Source at Kyutai, From Online Demos to On-Device
36mKeynote: Scientific Discovery: From the Lab Bench to the GPU
30mConversational AI: Building clever chatbots
40mKeynote: Open-source Multimodal AI in the Wild