Speech recognition for subtitles in VK Video
summary
Describes VKontakte’s speech recognition system used to produce subtitles for videos, explaining differences between recognizing short voice messages and long video audio. Covers the concept of a speech domain and why models can perform much worse than expected when audio mismatches that domain.
AI-written summary · about
chapters (26)
- 0:00Introduction
- 1:09Problem statement and requirements
- 2:18Acoustic features and spectrograms
- 5:07Acoustic models and voiceoids
- 7:59Model evolution: LSTM to modern CNN
- 9:24Modern architectures: QuartzNet, Conformers
- 11:52Training models and CTC loss
- 13:16Streaming vs offline recognition
- 14:20RNN-T: architecture and implementation challenges
- 16:25Language models and their role
- 17:30Tokenization approaches: word to character
- 18:32Decoding: Greedy vs Beam Search
- 22:23Integration issues and solutions
- 24:51Lexicon and informal speech problems
- 26:37Liner issues and solutions
- 28:50Quality metrics and test results
- 29:58API and model availability
- 31:03Announcement by Philipp Malikovsky
- 31:45Q&A: Model development time
- 33:26Q&A: Video processing time
- 34:37Q&A: Multiple speakers handling
- 36:57Q&A: Offline models on smartphones
- 37:39Q&A: Trimming and background noise
- 38:44Q&A: Emotions in Emoji
- 39:47Q&A: Domain-specific term recognition
- 41:47Q&A: Topic-based refinement
related talks
38mVKontakte Speech Recognition Engines
45mAuditory Intelligence : beyond speech recognition
56mSpeech Recognition: How to Build Speech-to-Text Yourself
22mDeep learning module recognizes video sounds
45mTranscend Language Barriers with Custom Machine Learning APIs: um...
01h 34mReinventing Speech-to-Text Transcriptions with Go and Whisper
50mSearching video content with AI
37mSpeech to text with Elasticsearch